Linux log files

Linux Log files and usage

=> /var/log/messages : General log messages

=> /var/log/boot : System boot log

=> /var/log/debug : Debugging log messages

=> /var/log/auth.log : User login and authentication logs

=> /var/log/daemon.log : Running services such as squid, ntpd and others log message to this file

=> /var/log/dmesg : Linux kernel ring buffer log

=> /var/log/dpkg.log : All binary package log includes package installation and other information

=> /var/log/faillog : User failed login log file

=> /var/log/kern.log : Kernel log file

=> /var/log/lpr.log : Printer log file

=> /var/log/mail.* : All mail server message log files

=> /var/log/mysql.* : MySQL server log file

=> /var/log/user.log : All userlevel logs

=> /var/log/xorg.0.log : X.org log file

=> /var/log/apache2/* : Apache web server log files directory

=> /var/log/lighttpd/* : Lighttpd web server log files directory

=> /var/log/fsck/* : fsck command log

=> /var/log/apport.log : Application crash report / log file

ssh takes a long time to connect or log in

Large delays (more than 10 seconds) are typically caused by a problem with name resolution:

• Some versions of glibc (notably glibc 2.1 shipped with Red Hat 6.1) can take a long time to resolve “IPv6 or IPv4″ addresses from domain names. This can be worked around with by specifying AddressFamily inet option in ssh_config.
• There may be a DNS lookup problem, either at the client or server. You can use the nslookup command to check this on both client and server by looking up the other end’s name and IP address. In addition, on the server look up the name returned by the client’s IP-name lookup. You can disable most of the server-side lookups by setting UseDNS no in sshd_config.

Ubuntu 10.04 vnc-based login server

Ubuntu 10.04 vnc-based login server

Enable Java Plugin on Firefox 3.6

Pre-requisites.

I don’t know how much of it its really needed, but I install the whole Sun’s Java 6 packages,

$ sudo apt-get install sun-java6-jdk sun-java-6-jre sun-java6-pluging
   sun-java6-source sun-java6-bin

For sure you’ll need the jdk and plugin one… If your internet connection is slow you might try installing these two first and try the above command.

Configuration of Java for Mozilla.

For  enable Java plugin on Firefox 3.6,

$ sudo update-alternatives --install /usr/lib/mozilla/plugins/libjavaplugin.so \
   mozilla-javaplugin.so /usr/lib/jvm/java-6-sun/jre/lib/i386/libnpjp2.so 50

Now it’s time to restart your browser.

For checking that the plugin is enabled, open your Firefox browser and type

about:plugins

in the address line, there must be the Java application enabled.



Query an IBM ServeRAID adapter on the CLI

I quickly needed to determine the status of a RAID array that sat on an IBM ServeRAID adapter. The boxen was running Ubuntu 8.04 x86_32, I didnt want to install RaidManager nor did I want to install a debian package. Here’s how.

$ wget http://hwraid.le-vert.net/debian/pool-lenny/arcconf_6.10.18451-1_i386.deb
$ ar -x arcconf_6.10.18451-1_i386.deb
$ tar zxvf data.tar.gz
$ sudo ./usr/sbin/arcconf GETCONFIG 1

you may need to install binutils package to run “ar”

Install ioncube in ubuntu

1. Download ionCube loaders

sudo wget http://downloads.ioncube.com/loader_downloads/ioncube_loaders_lin_x86.tar.gz


2. Extract

sudo tar zxvf ioncube_loaders_lin_x86.tar.gz


3. Move to a permanent location

sudo mv ioncube /usr/local/


4. Add reference to your php.ini file (sudo pico /etc/php5/apache2/php.ini)

zend_extension = /usr/local/ioncube/ioncube_loader_lin_5.2.so


There are a few versions of the loader in the tar archive. Use the one that matches your PHP version.

5. Restart apache

sudo /etc/init.d/apache2 restart

6. check installation using following command:
php -v
you should see following output:

PHP 5.1.6 (cli) (built: Apr 7 2009 08:00:04)
Copyright (c) 1997-2006 The PHP Group
Zend Engine v2.1.0, Copyright (c) 1998-2006 Zend Technologies
with the ionCube PHP Loader v3.1.34, Copyright (c) 2002-2009, by ionCube Ltd.

Cloning VMs on Linux Hosts with VMWare Server 2.0

think it’s a MAJOR pain not having a “clone” feature in VMWare Server without using Virtual Infrastructure, so I made my own Linux shell scripts to do it.

The clone script does a straight clone of a VM. It copies the directory, renames the files and tweaks all the text inside the files by using sed to replace the old name with the new one, then sets all the permissions to what they would be if you created a new VM.

The rename script basically does all but copy it. This is handy for if you tried to rename a directory manually or attempted a manual copy and then found it still shows up under the old name in the web console. Remember this one assumes you have already renamed the directory to the new name and it MUST be the same as what you intend for the new name to be.

Both work the same way… scriptname <old name> <new name>

You can use quotes around the names if there are spaces, I went to great lengths to make sure it would still work with spaces in names (although I personally hate spaces).

Save the scripts to the base directory where your VM’s are, “standard” location is “/var/lib/vmware/Virtual Machines”. Make executable with chmod +x <scriptname>.

Once ran, add the new/renamed VM to the web console and choose “I copied it” or “I moved it” as appropriate.

These scripts are offered free to everyone to use, modify, or do anything else you want with except take credit for the original version. Absolutely no warranty or guarantee of any kind on the part of anyone that they won’t hose your VMs or even your whole system.

If I understand correctly how the split disks work, the 3rd file attached (clone-split-disk.sh) should work for those VM’s. I have not tested it myself and will likely not get a chance anytime soon, so I will rely on feedback from the community for any near-future tweaking. This is really just a change to one line (line 34) in the file – in the original we exclude the binary .vmdk file from parsing to find-and-replace since this is how single-disk-file versions are saved; in this version we exclude -f???.vmdk where “?” is a number between 0-9 (i.e. files like -f001.vmdk, -f002.vmdk, etc) while we parse the .vmdk file since with split disks, the .vmdk is a text file describing the -f???.vmdk binary disk files. IF you use a combination of split disks and single-file disks, probably neither one of these will work – maybe in a future version I will be able to merge the two scripts and add logic to tell the disk types apart.

clone_vm
change_name
clone-split-disk

Setup Ubuntu + pureftpd + pureAdmin

# sudo -s
# apt-get install pure-ftpd pureadmin

Create the user and group accounts:

sudo groupadd ftpgroup
sudo useradd -g ftpgroup -d /dev/null -s /etc ftpuser

sudo mkdir /home/ftpusers
sudo chown -R ftpuser:ftpgroup /home/ftpusers

# pure-pw useradd joe -u ftpuser -d /home/ftpusers/joe
Type in the password when prompted for the new user twice.

Create the database and make sure that the pure-ftpd configuration strictly uses the pdb file for managing accounts:

# pure-pw mkdb
# ln -s /etc/pure-ftpd/conf/PureDB /etc/pure-ftpd/auth/PureDB
# gedit /etc/pure-ftpd/conf/PAMAuthentication
Change yes into no (or yes to allow system users to logon)

Make sure that a backup file (.PAMAuthentication~) isn’t created because it will create a conflict when trying to restart service. Using PAM can allow public access to administrative accounts using pure-ftpd, that’s why I disable it within pure-ftpd. I prefer it off so I can strictly use virtual accounts maintained within the pdb file.

Finally it’s time to enable the pdb file and yes you do have to stop the server in order to avoid errors stating that the port is already in use.

# /etc/init.d/pure-ftpd stop
# /usr/sbin/pure-ftpd -j -lpuredb:/etc/pure-ftpd/pureftpd.pdb &
# /etc/init.d/pure-ftpd start

It works great and is very simple to administer with PureAdmin. Most of the virtual user creation can be done there. Saves a lot of time. The visual log helps out a bit too.

Later you can use
#pure-ftpd-control restart
May need to add
# ln -s /etc/pure-ftpd/conf/PureDB /etc/pure-ftpd/auth/50PureDB to allow it to point to .pdb

Restarting ftp server: Running: /usr/sbin/pure-ftpd -l puredb:/etc/pure-ftpd/pureftpd.pdb -u 1000 -E -O clf:/var/log/pure-ftpd/transfer.log -j -B

Replacing A Failed Hard Drive In A Software RAID1/5 Array

Replacing A Failed Hard Drive In A Software RAID1 Array

Version 1.0
Author: Falko Timme <ft [at] falkotimme [dot] com>
Last edited 01/21/2007

This guide shows how to remove a failed hard drive from a Linux RAID1/5 array (software RAID), and how to add a new hard disk to the RAID1/5 array without losing data.

Take RAID1 as an example.

1 Preliminary Note

I have two hard drives, /dev/sda and /dev/sdb, with the partitions /dev/sda1 and /dev/sda2 as well as /dev/sdb1 and /dev/sdb2.

/dev/sda1 and /dev/sdb1 make up the RAID1 array /dev/md0.

/dev/sda2 and /dev/sdb2 make up the RAID1 array /dev/md1.

/dev/sda1 + /dev/sdb1 = /dev/md0

/dev/sda2 + /dev/sdb2 = /dev/md1

/dev/sdb has failed, and we want to replace it.

2 How Do I Tell If A Hard Disk Has Failed?

If a disk has failed, you will probably find a lot of error messages in the log files, e.g. /var/log/messages or /var/log/syslog.

You can also run

cat /proc/mdstat

and instead of the string [UU] you will see [U_] if you have a degraded RAID1 array.

3 Removing The Failed Disk

To remove /dev/sdb, we will mark /dev/sdb1 and /dev/sdb2 as failed and remove them from their respective RAID arrays (/dev/md0 and /dev/md1).

First we mark /dev/sdb1 as failed:

mdadm –manage /dev/md0 –fail /dev/sdb1

The output of

cat /proc/mdstat

should look like this:

server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0] sdb1[2](F)
24418688 blocks [2/1] [U_]

md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/2] [UU]

unused devices: <none>

Then we remove /dev/sdb1 from /dev/md0:

mdadm –manage /dev/md0 –remove /dev/sdb1

The output should be like this:

server1:~# mdadm –manage /dev/md0 –remove /dev/sdb1
mdadm: hot removed /dev/sdb1

And

cat /proc/mdstat

should show this:

server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0]
24418688 blocks [2/1] [U_]

md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/2] [UU]

unused devices: <none>

Now we do the same steps again for /dev/sdb2 (which is part of /dev/md1):

mdadm –manage /dev/md1 –fail /dev/sdb2

cat /proc/mdstat

server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0]
24418688 blocks [2/1] [U_]

md1 : active raid1 sda2[0] sdb2[2](F)
24418688 blocks [2/1] [U_]

unused devices: <none>

mdadm –manage /dev/md1 –remove /dev/sdb2

server1:~# mdadm –manage /dev/md1 –remove /dev/sdb2
mdadm: hot removed /dev/sdb2

cat /proc/mdstat

server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0]
24418688 blocks [2/1] [U_]

md1 : active raid1 sda2[0]
24418688 blocks [2/1] [U_]

unused devices: <none>

Then power down the system:

shutdown -h now

and replace the old /dev/sdb hard drive with a new one (it must have at least the same size as the old one – if it’s only a few MB smaller than the old one then rebuilding the arrays will fail).

4 Adding The New Hard Disk

After you have changed the hard disk /dev/sdb, boot the system.

The first thing we must do now is to create the exact same partitioning as on /dev/sda. We can do this with one simple command:

sfdisk -d /dev/sda | sfdisk /dev/sdb

You can run

fdisk -l

to check if both hard drives have the same partitioning now.

Next we add /dev/sdb1 to /dev/md0 and /dev/sdb2 to /dev/md1:

mdadm –manage /dev/md0 –add /dev/sdb1

server1:~# mdadm –manage /dev/md0 –add /dev/sdb1
mdadm: re-added /dev/sdb1

mdadm –manage /dev/md1 –add /dev/sdb2

server1:~# mdadm –manage /dev/md1 –add /dev/sdb2
mdadm: re-added /dev/sdb2

Now both arays (/dev/md0 and /dev/md1) will be synchronized. Run

cat /proc/mdstat

to see when it’s finished.

During the synchronization the output will look like this:

server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0] sdb1[1]
24418688 blocks [2/1] [U_]
[=>...................]  recovery =  9.9% (2423168/24418688) finish=2.8min speed=127535K/sec

md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/1] [U_]
[=>...................]  recovery =  6.4% (1572096/24418688) finish=1.9min speed=196512K/sec

unused devices: <none>

When the synchronization is finished, the output will look like this:

server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0] sdb1[1]
24418688 blocks [2/2] [UU]

md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/2] [UU]

unused devices: <none>

That’s it, you have successfully replaced /dev/sdb!

Upgrade Ubuntu Dapper / Edgy / Hardy

Today you might have release Ubuntu 6.04 or 6.1 .The common name given to this release from the time of its early development was Dapper or Edgy.