The Tech Diary

The following function definition creates a split() function that splits a string by a given delimiter and returns the specified element.

DROP FUNCTION IF EXISTS split;
CREATE FUNCTION split (str TEXT, delim VARCHAR(1), N INT)
RETURNS TEXT DETERMINISTIC
RETURN SUBSTRING(SUBSTRING_INDEX(str, delim, N), LENGTH(SUBSTRING_INDEX(str, delim, N - 1)) + LENGTH(delim) + (N > 1))

The syntax of the function is as follows.

split(str, separator, pos)

Comment (0)

There's several ways to count the number of characters in a file within vim.

The 'n' modifier to the substitute operator ('s') tells vim only to report the number of matches, but not to actually do any substitutions.  Given that, the following command will report the number of characters in our file:

:%s/.//gn

That says, "find ('s///n') every occurrence ('g') of any character ('.') in all lines ('%')."  Note that this will not include any line break characters.  To count line breaks, just prepend the '_' modifier to the '.', like so:

:%s/_.//gn

You can also count the characters in a specified range by using a range instead of the '%'.  To count all characters, including line breaks, from lines 20 to 80 do this:

:20,80s/_.//gn

Happy vimming!

Comment (0)

I've been plagued with a recurring problem with dspam that I had been unable to track down for months.  While dspam catches nearly every spam that comes into my Inbox, I noticed a pattern of email that not only weren't flagged as spam, they apparently weren't even checked by dspam; there were no X-DSPAM headers in the email.

I spent hours tracking down whether there were any qmail aliases that were dropping mail directly into users Maildirs, since that would explain what I was seeing, but there were none to be found.

I then turned on tcpserver logging to see if maybe these spams were coming from a previously whitelisted IP in my tcp.smtp.  Nothing there either.

After all of this, I happened to notice an error in my qmail logs:

delivery 76475: success: 19066:_[10/05/2009_13:02:17]_message_too_big,_delivering/did_0+0+1/

I dunno why I didn't notice that before, but that error explained the problem.  dspam has a setting in dspam.com called "MaxMessageSize".  Any emails larger than this are passed through without any spam checking.  The idea is that you don't want dspam slowing down your mail server by choking through emails with 200MB binary attachments.  In my case, MaxMessageSize was set to 300KB, and sure enough, most of the spams getting through were larger than 300KB. After bumping it up to 2MB, nearly all of them have been stopped.

Comment (0)

AUser Manager is a fantastic component that adds much-needed security to the Joomla! user registration process.  It transparently adds full-featured AJAX Captcha and RBL tests, all without any hacks or modifications to core Joomla! files.

I've attached a slightly modified version of 1.5.10 that has the following "improvements":

• AUser Manager parameter options to disable the strong password generator, username available AJAX notice & password meter.  They're enabled by default, but it gives the admin the option to make com_auser look more like com_user if they wish.
• en-US language file that's a bit more native for those of us on this side of the pond.
• Audio digits and letters taken from Pat Fleet's (the voice of AT&T) freeware IVR prompts for asterisk.  They're easier to understand than the computer-generated mp3s.
• Minor spelling changes (my obsessive-compulsive nature kicked in on these)

Hope my effort adds to this already great component.  Long live Joomla!

Download

Comment (0)

To insert line numbers at the beginning of every line in Vim, do this:

:%s/^/=line(".")

This is a global search and replace (%s/[search]/[replace]), matching the beginning of each line (^) with an expression (=) that returns the current line number (line(".")).

To add a comma (or any other character) after the line number, do it like this:

:%s/^/=line(".") . ","

Any valid Vim expression is valid after the "=", so you could start line number with 100 like this:

:%s/^/=100+line(".")

Pretty simple and straightforward, when you think about it.

Comment (0)

syslog is the standard system logging tool on most flavors of unix.  It stores records of system events in a number of user-defined files.  On FreeBSD, these files live by default in /var/log.  syslog has a configuration file, /etc/syslog.conf, that determines which file to log system events.

What a lot of people don't know is that syslog can send events to a remote syslog server.  This means you can have all of your logs from multiple machines stored on a single host automatically.  Both the client and the server need properly configured config files.

Here's a quick primer on syslog.conf:

The file is broken into sections separated by program and/or hostname definition lines.  Program definitions are in the form of "!program" (ex: !httpd), while hostname definitions take the form "+hostname" (ex: +10.0.1.1).  "-" (minus) can be used to exclude a program or hostname, so "!-httpd" would log every program except httpd.  Multiple definitions can be separated by commas (ex: !httpd, qmail).  Program and hostname definitions can be reset by using "*" (!* or +*).

Each section contains rules that are valid for only the most recent program and hostname definitions.  Rules are made up of two fields, the selector and the action, separated by tabs or spaces.  Selectors define what types of messages and priorities and the action defines what to do with those messages.

Selectors look like this: facility.level, where facility is the part of the system that generated the message and can be one of the following keywords:

auth, authpriv, console, cron, daemon, ftp, kern, lpr, mail, mark, news, ntp, security, syslog, user, uucp and local0 through local7.

Multiple selectors can be separated by a ';'.

Level defines the severity of the message and can be one of the following keywords:

emerg, alert, err, warning,  notice, info and debug.

You can also use the comparison flags !, <, > and = after the "." to specify a range of severity levels.  For example, "mail.<=notice" would log all notice, info and debug messages for the mail facility.  The default flag is >=, so "mail.alert" would log alert and emerg messages for mail.

The action field can take one of five forms:

• the full path of a file to which the message is appended (/var/log/mail.log)
• a hostname on which a syslog server is listening (@hostname)
• a comma-separated list of users who will see messages on their terminal if logged in (root, ecarter)
• a *, which writes the message to all logged-in users
• a | followed by a command (| mail admin)

So how do you combine these configuration parameters to set up remote syslog?  Let's look at a simple example with a server named "logmaster" and a client named "mailhost".

Server Config

# Log all messages coming from mailhost
+mailhost
*.*                                          /var/log/mailhost/mail.log

Client Config

# local logging
*.notice;kern.debug;lpr.info       /var/log/messages

# send all mail messages to logmaster
mail.*                                     @logmaster

This is a basic remote syslog configuration that simply sends all messages on mailhost with a "mail" facility to logmaster, where they are appended to a file.  This isn't necessarily idea since all the mail messages, regardless of their originating program, are globbed into a single file.

By using the program definition after the host definition in the syslog.conf on the server, we can separate log messages based on which program they came from.

Server Config

# Log all messages coming from mailhost
+mailhost

# log qmail messages of all facilities and severities
!qmail
*.*                                          /var/log/mailhost/qmail.log

# log messages from imapd of all facilities and severities
!imapd
*.*                                          /var/log/mailhost/imapd.log

# log messages from POP daemon of all facilities and only err+ severity
!pop3d
*.err                                        /var/log/mailhost/pop3d.log

In this config, we're taking all the mail syslog messages from mailhost and saving ones from qmail, imapd and pop3d to their respective files.

You can probably see how using different combinations of the program and hostname definitions and syslog facilities can match any message and direct it where you wish.

That's remote syslog logging in a nutshell.

 

Comment (0)

Here's a couple of ways to do that:

• Alt-F4 brings up the standard shutdown dialog.
• From a command prompt type: shutdown -r -t 0

Comment (0)

One of the first things I do when setting up a new Windows machine is enable the option to "Show hidden files and folders" and disable the option to "Hide protected operating system files (Recommended)".  You can do this in the Folder Options control panel on the View tab.  Then I can see everything on my drive, even the stuff Microsoft thinks is too dangerous for me to see.

On Win 2K and XP, this works great, but in Vista there's a little snag.  The Desktop.ini file shows up on the desktop.  Not only that, there's two copies that show up!

Here's how to fix that:

* Open windows explorer
* Select the desktop in the "folders" tree view
* From the "Organize" menu choose "Folder and Search Options"
* Click the "View" tab
* Check the "Do not show hidden files and folders" (this ONLY applies to the desktop folder)

Comment (0)

When I was trying to troubleshoot some XP startup issues, I decided to rename the Start->Programs->Startup folder to "_Startup", thinking it would disable any startup programs.

Turns out the folder is somehow flagged as a startup folder, and no matter what it's called, the programs will still start up.  In addition, Windows will create a new Startup folder.  Now I've got both a "Startup" and a "_Startup" in my Programs.  I can't delete either one since Windows complains with:

<FOLDER> is a Windows system folder and is required for Windows to run properly. It can not be deleted.

To fix this, I had to move the _Startup folder, rename it, then move it back.  The folder actually reside in "C:\Documents and Settings\[user]\Start Menu\Programs".   I dragged _Startup up to the "Start Menu" folder, renamed it back to "Startup", then dragged it back to "Programs".  I clicked okay when Windows asked if I wanted to copy on top of the current folder.

Comment (0)

After adding the HighPoint RocketRaid software package to FreeBSD, I'm now getting the following errors whenever I add or delete a package or port:

pkg_add: package hptraidconf-3.2 has no origin recorded
pkg_add: package hptsvr-3.13 has no origin recorded

The "origin" is normally the directory in the ports tree from which the package was built.  However, the 3rd-party HighPoint packages aren't part of the standard FreeBSD ports tree, and so the pkg_* utilities complain about a missing entry in the package database (/var/db/pkg).  Here's the first few lines of the +CONTENTS file in /var/db/pkg/ipmitool-1.8.11:

@comment PKG_FORMAT_REVISION:1.1
@name ipmitool-1.8.11
@comment ORIGIN:sysutils/ipmitool
@cwd /usr/local
bin/ipmitool 

Even though I added impitool with 'pkg_add -r', you can see the ORIGIN is listed as sysutils/ipmitool.

So, how do we fix this?  The 'pkgdb' tool has an option to fix the package database.  pkgdb is part of the portupgrade package, so install that first:

pkg_add -r portupgrade

-OR-

cd /usr/ports/ports-mgmt/portupgrade; make install

Now we can run pkgdb with the -F (fix) option:

# pkgdb -F
---> Checking the package registry database
[Rebuilding the pkgdb in /var/db/pkg ... - 10 packages found (-0 +10) .......... done]
Missing origin: hptraidconf-3.2
[Updating the portsdb in /usr/ports ... - 20037 port entries found .........1000.........2000.........3000.........4000.........5000.........6000.........7000.........8000.........9000.........10000.........11000.........12000.........13000.........14000.........15000.........16000.........17000.........18000.........19000.........20000 ..... done]
Skip this for now? [yes] no
Guessing... no idea.
Not in due form :
New origin? (? to help): ?
[Enter] to skip, [Ctrl]+[D] to unregister or deinstall,
[.][Enter] to abort, [Tab] to complete
New origin? (? to help): sysutils/hptraidconf
sysutils/hptraidconf: Not found. Force it? [no] yes
(? hptraidconf-3.2)Fixed. (-> sysutils/hptraidconf)
Missing origin: hptsvr-3.13
Skip this for now? [yes] no
Guessing... no idea.
Not in due form :
New origin? (? to help): sysutils/htpsvr
sysutils/htpsvr: Not found. Force it? [no] yes
(? hptsvr-3.13)Fixed. (-> sysutils/htpsvr)

pkgdb again informs us that there's no origin for hptraidconf and hptsvr and tries to find the proper origin.  Because these packages aren't part of the ports tree, it has "no idea", and asks us to provide one.  I chose to say they were part of the sysutils tree, but it really doesn't matter.  After entering a value, pkgdb warns that it can't find that port so we have to force the origin.

After this, all pkg_* utilities work without complaining.

 

Comment (0)