Posted by aionman on Jun 29, 2011 in
Linux http://distilledb.com/blog/archives/date/2009/02/22/swap-files-in-linux.page
Understanding swap files in Linux
Summary If you run anything close to a modern operating system, you almost certainly interact with a swap file. You might be familiar with the basics of how these work: they allow your OS to prioritize more frequently-used pages in main memory and move the less frequently-used ones to disk. But there’s a lot going on underneath the covers. Here’s a simple guide to swap files in Linux.
To appease some of my hungrier applications and support heftier development efforts, I recently upgraded the memory on my system from 4 GiB to 8 GiB. That gave me occasion to tinker around with the swapping behavior of my system and check things out.
If you run anything close to a modern operating system, you almost certainly interact with a swap file. You might be familiar with the basics of how these work: they allow your OS to prioritize more frequently-used pages in main memory and move the less frequently-used ones to disk. But there’s a lot going on underneath the covers. Here’s a simple guide to the theory and practice of swap files in Linux, and how you can tweak things for your benefit.
An abstract memory model
It’ll be useful to have a mental model of the way memory works in general1, so we’ll start from the basis of a simple one here.
In general, all computers have access to physical memory, where the actual bits are manipulated and stored for use. Most modern operating systems present physical memory to higher-level applications as an abstraction calledvirtual memory. This allows applications to see memory as if it were a contiguous block, even though the underlying physical memory may theoretically be taken from many arbitrary, heterogeneous sources — multiple memory chips, flash memory, disk drives, and so on.
The memory manager divides virtual memory into chunks of identical size called pages. A page is the smallest amount that the OS will allocate in response to requests from programs. It is also the smallest unit of transfer between main memory and any other location, such as a hard disk. The size of a page is usually fixed by the operating system’s kernel2.
Applications see virtual memory as a contiguous resource divided into units called pages. A typical page size for a modern desktop system is about 4 kilobytes.
As pages are allocated to applications, they are assigned pages in the physical memory space through a special mapping called address translation. Applications don’t know where they are in the physical space; they see only the pages they use.
Address translation maps virtual pages onto physical pages. This mapping is transparent to applications.
The memory manager knows how to aggregate different backing stores to provide the abstraction of contiguous virtual memory. By updating the address translation mechanism so that a virtual page always points to the correct physical page, the memory manager may move pages around in physical memory without directly impacting applications.
The backing stores for pages can be quite heterogeneous.
Wide variation in different kinds of physical memory
Not every backing store displays the same storage and speed characteristics. If different types of physical memory display different characteristics, the memory manager can exploit these differences to optimize system performance. It can make intelligent decisions about which pages should go where.
Consider the difference between main memory and a hard drive, for example.
| characteristic |
storage medium |
| on disk |
in main memory |
| absolute |
relative |
absolute |
relative |
| access time |
5ms |
1× |
10ns |
500,000× faster |
| peak transfer rate |
80 MB/s |
1× |
8 GB/s |
100× faster |
| storage space |
100 GB |
25× larger |
4 GB |
1× |
| price/GB storage |
$0.60/GB |
20× cheaper |
$12/GB |
1× |
Some comparisons of a typical hard drive and typical main memory on a consumer-grade desktop.
Because storage on disk-backed file systems does indeed have different characteristics than storage in main memory, there’s significant room to optimize depending on the characteristics of how pages are accessed. As the table shows, hard disks are several orders of magnitude slower and less efficient at retrieving data than main memory. Thus, the memory manager needs to carefully balance the demand for memory against the different kinds of supply.
Flavors of pages
If all pages were of the same kind, deciding which pages should go where would be a simpler decision. For example, one strategy is to make the access time quickest for the pages that are accessed the most frequently. Unfortunately, it’s not just the types of memory that are heterogeneous, but also the contents of the pages that are stored there. On Linux, there are four different kinds of pages.
- Kernel pages. Pages holding the program contents of the kernel itself. Unlike the other flavors, these are fixed in memory once the operating system has loaded and are never moved.
- Program pages. Pages storing the contents of programs and libraries. These are read-only, so no updates to disk are needed.
- File-backed pages. Pages storing the contents of files on disk. If this page has been changed in memory (for example, if it’s a document you’re working on), it will eventually need to be written out to disk to synchronize the changes.
- Anonymous pages. Pages not backed by anything on disk. When a program requests memory be allocated to perform computations or record information, the information resides in anonymous pages.
When the in-memory version of a page is the same as the one on disk, we say that the page is clean; the contents are the same. But sometimes the contents of a page have been updated since the last time they were read. When this happens, the page becomes dirty.
A clean page can be repurposed for something else easily; no updates need to be made, and the page can simply be recycled. But a dirty page has to be written back to disk before it can be used again. For file pages, this is an expensive operation, so the kernel tries to avoid the overhead of flushing back to disk when it can.
For anonymous pages, there’s a different problem. Effectively, they’re always dirty: the very act of creating the anonymous page means that there is now data that is in memory which isn’t in disk. If the kernel wants to use anonymous pages for something else, it must first reclaim them. But anonymous pages have no files to back them. How can you flush something back to disk when there’s nowhere to flush it to?
Swap files
The use of swap can resolve many of these issues. Swap is a disk-backed area that’s treated as an extension of main memory. It serves as a holding area for pages that have been evicted by the kernel. Let’s use an illustrative example to show how swap files help make memory work better.
Legend for the next few diagrams. Unused pages have dotted borders; dirty pages have an alert symbol; and anonymous and file pages are colored orange and green, respectively.
When a moderately loaded system gets additional requests for memory, the kernel generally draws from the pool of free pages first to fulfill these requests. If there are few free pages remaining, the kernel tries to flush clean pages to make room for the new requests.
The kernel prefers to go after unused pages first.
If the clean pages also become depleted, the kernel is forced to clean a dirty page and then flush it. This is an expensive operation. For this reason, the kernel tries to maintain at least some clean pages all the time.
When the ratio of anonymous pages to dirty pages is high and the number of clean pages is low, the kernel is running out of memory. Without swap, this situation will require a number of costly disk writes. Consider a request for allocation when a number of dirty pages are already present.
Without swap space, it’s easier for systems to get overloaded.
In the figure above, a request for two anonymous pages has come in. There are no more unused pages, so the kernel must drop one of the existing pages to satisfy the request. The kernel can use one page freely: the single clean file page in slot 6.
But to allocate the second page, the kernel now has to flush one of the dirty file pages (in slots 1, 3, or 4) back to disk to make room. It cannot move the page in slot 5 anywhere, because it is anonymous and has no backing store; there’s nowhere else to put it. Even if this page has not been used in a very long time, it must still occupy space in memory until the process using it has released the page.
When space is tight and there’s no swap the kernel must make room by cleaning dirty pages and freeing them. In this example, the kernel is forced to clean page #1 back to disk to make room for the second allocated page.
Without swap, the kernel gets boxed into this unfortunate corner more easily.
With swap, however, the kernel gets an additional tool to use in its arsenal. Instead of being forced to clean one of the dirty pages, it can instead evict one of the anonymous pages to the swap region.
When swap is available, the kernel doesn’t need to clean dirty pages, and can instead move anonymous pages to swap.
As in the earlier non-swap scenario, the kernel use can use the clean page in slot 6 for the first requested page. It is allocated and the clean page is dropped.
For the second requested page, the kernel must no longer clean a dirty page to make room. Instead, it can simply flush one of the anonymous pages to the swap region. The code required to do this is generally very simple and significantly less complex than cleaning a dirty page, and the kernel prefers swapping to cleaning dirty file-backed pages.
Optimizing your swap settings
Linux provides a number of ways to interact with your swap. Two are detailed here:
- Aggressiveness of swapping
- Adding and removing additional swap containers
Controlling aggressiveness of swapping
The more aggressively the kernel swaps, the more efficiently existing memory can be put to use. Pages that look like they’re not being used will be swapped out rapidly. If the kernel swaps too often, though, applications that were using those pages will take longer to become responsive again as the kernel swaps their memory back into main memory.
For a desktop user, responsiveness of applications can be important, so an aggressive swap may not be desirable, even if it results in less efficient use of memory. For servers and other non-interactive systems, more aggressive swapping may be appropriate and acceptable.
On Linux, this careful balancing act can be configured to meet your personal preferences. The kernel swaps out pages with a zealousness controlled by a swappiness setting.
Swappiness is an integer that ranges from 0 to 100, and indicates the degree to which the kernel favors swap space over main memory. Higher swappiness means that the kernel will move things to swap more frequently. Lower swappiness means that the kernel tries to avoid using swap. A swappiness of zero causes the kernel to avoid swap for as long as possible.
Ubuntu and several other Linux distributions have a default swappiness of 60. You can check your swap setting by reading a /proc/sys value:
$ cat /proc/sys/vm/swappiness
60
To temporarily modify your swappiness, simply edit this value:
$ sudo sysctl vm.swappiness=40
vm.swappiness = 40
This setting lasts until reboot or you change it again with another sysctl vm.swappiness invocation. To make this setting take effect on every reboot, edit your /etc/sysctl.conf configuration file.
$ gksudo gedit /etc/sysctl.conf
Find the vm.swappiness line; if none exists, add it.
vm.swappiness = 40
Adding swap containers
Modern operating systems generally have either a swap partition or a swap file. In a swap partition, part of the hard drive is sliced off and becomes dedicated to swap. A swap file is just an ordinary file that holds up to its file size in swapped pages.
A swap file is considerably less complicated than a swap partition to establish. There is no speed difference between the two3, so swap files are favorable in this respect. However, if you want to be able to hibernate or suspend your computer, using a swap partition is required in some cases. (These suspend/hibernate managers usually cannot handle writing to an active file system.)
Making a new swap file is a simple process. In this example, we’ll make a 2 GiB swap file and make it available to the system as additional swap space. We’ll use primary.swap as the name of the example swap file, but there is nothing special about the name of the file or its extension. You may use anything you wish.
First, we need to create the swap file itself. We’ll use a stream of zeroes as the input source (if=/dev/zero), and write it out to a file named primary.swap in the /mnt directory (of=/mnt/primary.swap). We will write 2048 (count=2048) blocks each 1 MiB in size (bs=1M). Depending on the speed of your hard disks, this may take a little while.
$ sudo dd if=/dev/zero of=/mnt/primary.swap bs=1M count=2048
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 30.1085 s, 71.3 MB/s
Next, we need to format this file and prepare it for use as a swapping space. The mkswap utility sets up a swap area on a device or file.
$ sudo mkswap /mnt/primary.swap
Setting up swapspace version 1, size = 2097148 KiB
no label, UUID=7be2b3b6-83b0-4afd-8537-197cf12f8c59
After formatting it, the swap can now be added to our system. Use the swapon utility to activate the swap region.
$ sudo swapon /mnt/primary.swap
You can verify that your swap space is now 2 GiB larger.
$ cat /proc/meminfo | grep SwapTotal
SwapTotal: 2097144 kB
Your changes will be lost at reboot, so if you want to make them permanent we’ll need to edit your filesystem table in /etc/fstab.
$ gksudo gedit /etc/fstab
Now add your swap file to the list of filesystems to mount at boot by appending a line to the file.
/mnt/primary.swap none swap sw 0 0
Removing a swap file
Removal works much the same way, but in reverse. If you’ve added your swap to the /etc/fstab list, you need to remove it here first.
To disable your running swaps, run the swapoff utility. You can either specify the swap you’d like to disable, or use the -a parameter.
$ sudo swapoff /mnt/primary.swap
When you disable swap, you force the kernel to clean every page on the swap and/or push it back to main memory. If there is not enough space to squeeze everything in, you may receive out of memory errors from the kernel, so use this judiciously.
Conclusion
Swap files are an essential part of the memory-management modules of operating systems. In Linux, adding and removing swap partitions and files is simple, and you can control how the kernel interacts with swap through configurable parameters. Through the use of these and other techniques, and with an understanding of the basics of swap, you can tweak your system’s use of memory to your heart’s content.
Additional reading
- Speed up your system by avoiding the swap file. FOSSWire. Accessed February 8th, 2009.
- 2.6 swapping behavior. LWN.net. Accessed February 8th, 2009.
- Swap FAQ. Ubuntu Documentation. Accessed February 12th, 2009.
- Patterson, David A. and Hennessy, John L. Computer Organization and Design: The hardware/software interface. © 1997. Morgan Kaufmann Publishers, San Francisco, California.
1 As this is only a model, we will naturally be leaving off some of the important but messier details.
2 On Linux, you can check your page size with the getconf command, specifying the PAGE_SIZE parameter. This returns the number of pages in bytes. 4 kilobytes (4,096 bytes) is a typical result on the x86 and x86_64architectures, so there are 256 pages / MB.
$ getconf PAGE_SIZE
4096
3 See this Linux kernel mailing list post.
Posted by aionman on Jun 29, 2011 in
Linux,
Ubuntu Linux Log files and usage
=> /var/log/messages : General log messages
=> /var/log/boot : System boot log
=> /var/log/debug : Debugging log messages
=> /var/log/auth.log : User login and authentication logs
=> /var/log/daemon.log : Running services such as squid, ntpd and others log message to this file
=> /var/log/dmesg : Linux kernel ring buffer log
=> /var/log/dpkg.log : All binary package log includes package installation and other information
=> /var/log/faillog : User failed login log file
=> /var/log/kern.log : Kernel log file
=> /var/log/lpr.log : Printer log file
=> /var/log/mail.* : All mail server message log files
=> /var/log/mysql.* : MySQL server log file
=> /var/log/user.log : All userlevel logs
=> /var/log/xorg.0.log : X.org log file
=> /var/log/apache2/* : Apache web server log files directory
=> /var/log/lighttpd/* : Lighttpd web server log files directory
=> /var/log/fsck/* : fsck command log
=> /var/log/apport.log : Application crash report / log file
Posted by aionman on Jan 20, 2011 in
Linux,
Ubuntu Large delays (more than 10 seconds) are typically caused by a problem with name resolution:
- Some versions of glibc (notably glibc 2.1 shipped with Red Hat 6.1) can take a long time to resolve “IPv6 or IPv4″ addresses from domain names. This can be worked around with by specifying AddressFamily inet option in ssh_config.
- There may be a DNS lookup problem, either at the client or server. You can use the
nslookup command to check this on both client and server by looking up the other end’s name and IP address. In addition, on the server look up the name returned by the client’s IP-name lookup. You can disable most of the server-side lookups by setting UseDNS no in sshd_config.
Posted by aionman on Dec 1, 2010 in
Linux,
Networking Installation Notes
Broadcom tg3 Linux Driver
Version 3.57b
04/28/2006
Broadcom Corporation
16215 Alton Parkway,
Irvine, CA 92619-7013
Copyright (c) 2004, 2005, 2006 Broadcom Corporation
All rights reserved
Table of Contents
=================
Introduction
Limitations
Packaging
Installing Source RPM Package
Building Driver From TAR File
Driver Settings
Driver Defaults
Unloading and Removing Driver
Driver Messages
Introduction
============
This file describes the tg3 Linux driver for the Broadcom NetXtreme
10/100/1000 Mbps PCI/PCI-X/PCI Express Ethernet Network Controllers.
The latest driver is in the latest 2.6 Linux kernel. It can also be
downloaded from http://www.broadcom.com as a source package, but is
generally not necessary to do so if you are using the latest 2.6
upstream kernel from http://www.kernel.org or one of the latest
vendor kernels from Red Hat, SuSE, or others.
The tg3 driver from the Broadcom package is almost identical to the
tg3 driver in the latest 2.6 upstream Linux kernel. It includes some
additional kernel compatible code to allow it to compile on older 2.6
and some 2.4 kernels. The version number is also similar but generally
has a one letter suffix at the end, (e.g. 3.55b) to distinguish it from
the in-kernel tg3 driver.
The next few sections on packaging, compiling, and installation apply
mostly to the Broadcom driver package only.
Limitations
===========
The current version of the driver has been tested on 2.4.x kernels starting
from 2.4.24 and all 2.6.x kernels. The driver may not compile on kernels
older than 2.4.24. Testing is concentrated on i386 and x86_64 architectures.
Only limited testing has been done on some other architectures such as
powerpc and sparc64.
Minor changes to some source files and Makefile may be needed on some
kernels.
Packaging
=========
To replace an older previously installed or in-kernel tg3 driver, follow
the instructions below.
The driver package from http://www.broadcom.com is released in two packaging
formats: source RPM and compressed tar formats. The file names for the two
packages are tg3-<version>.src.rpm and tg3-<version>.tar.gz respectively.
Identical source files to build the driver are included in both packages.
Installing Source RPM Package
=============================
The following are general guidelines for installing the driver.
1. Install the source RPM package:
rpm -ivh tg3-<version>.src.rpm
2. CD to the RPM path and build the binary driver for your kernel:
cd /usr/src/{redhat,OpenLinux,turbo,packages,rpm ..}
rpm -bb SPECS/tg3.spec
or
rpmbuild -bb SPECS/tg3.spec (for RPM version 4.x.x)
Note that the RPM path is different for different Linux distributions.
3. Install the newly built package (driver and man page):
rpm -ivh RPMS/<arch>/tg3-<version>.<arch>.rpm
<arch> is the architecture of the machine, e.g. i386:
rpm -ivh RPMS/i386/tg3-<version>.i386.rpm
Note that the –force option may be needed on some Linux distributions
if conflicts are reported.
The driver will be installed in the following path:
2.4.x kernels:
/lib/modules/<kernel_version>/kernel/drivers/net/tg3.o
2.6.x kernels:
/lib/modules/<kernel_version>/kernel/drivers/net/tg3.ko
4. Load the driver:
insmod tg3.o
or
insmod tg3.ko (on 2.6.x kernels)
or
modprobe tg3
5. To configure network protocol and address, refer to various Linux
documentations.
Building Driver From TAR File
=============================
The following are general guidelines for installing the driver.
1. Create a directory and extract the files:
tar xvzf tg3-<version>.tar.gz
2. Build the driver tg3.o (or tg3.ko) as a loadable module for the
running kernel:
cd src
make
3. Test the driver by loading it:
insmod tg3.o
or
insmod tg3.ko (on 2.6.x kernels)
or
insmod tg3
4. Install the driver:
make install
See RPM instructions above for the location of the installed driver.
5. To configure network protocol and address, refer to various Linux
documentations.
Driver Settings
===============
This and the rest of the sections below apply to both the in-kernel tg3
driver and the tg3 driver package from Broadcom.
Driver settings can be queried and changed using ethtool. The latest ethtool
can be downloaded from http://sourceforge.net/projects/gkernel if it is not
already installed. The following are some common examples on how to use
ethtool. See the ethtool man page for more information. ethtool settings do
not persist across reboot or module reload. The ethtool commands can be put
in a startup script such as /etc/rc.local to preserve the settings across a
reboot.
1. Show current speed, duplex, and link status:
ethtool eth0
2. Change speed, duplex, autoneg:
Example: 100Mbps half duplex, no autonegotiation:
ethtool -s eth0 speed 100 duplex half autoneg off
Example: Autonegotiation with full advertisement:
ethtool -s eth0 autoneg on
Example: Autonegotiation with 100Mbps full duplex advertisement only:
ethtool -s eth0 speed 100 duplex full autoneg on
3. Show flow control settings:
ethtool -a eth0
4. Change flow control settings:
Example: Turn off flow control
ethtool -A eth0 autoneg off rx off tx off
Example: Turn flow control autonegotiation on with tx and rx advertisement:
ethtool -A eth0 autoneg on rx on tx on
Note that this is only valid if speed is set to autonegotiation.
5. Show offload settings:
ethtool -k eth0
6. Change offload settings:
Example: Turn off TSO (TCP segmentation offload)
ethtool -K eth0 tso off
7. Get statistics:
ethtool -S eth0
8. Perform self-test:
ethtool -t eth0
Note that the interface (eth0) must be up to do all tests.
9. See ethtool man page for more options.
Driver Defaults
===============
Speed : Autonegotiation with all speeds advertised
Flow control : Autonegotiation with rx and tx advertised
MTU : 1500 (range 46 – 9000)
Some chips do not support jumbo MTUs bigger than
1500
Rx Ring Size : 200 (range 0 – 511)
Some chips are fixed at 64
Rx Jumbo Ring Size : 100 (range 0 – 255)
Not all chips support the jumbo ring, and some
chips that support jumbo frames do not use the
jumbo ring.
Tx Ring Size : 511 (range (MAX_SKB_FRAGS+1) – 511)
MAX_SKB_FRAGS varies on different kernels and
different architectures. On a 2.6 kernel for
x86, MAX_SKB_FRAGS is 18.
Coalesce rx usecs : 20 (range 0 – 1023)
Coalesce rx usecs irq : 20 (range 0 – 255)
Coalesce rx frames : 5 (range 0 – 1023)
Coalesce rx frames irq : 5 (range 0 – 255)
Coalesce tx usecs : 72 (range 0 – 1023)
Coalesce tx usecs irq : 20 (range 0 – 255)
Coalesce tx frames : 53 (range 0 – 1023)
Coalesce tx frames irq : 5 (range 0 – 255)
Coalesce stats usecs : 1000000 (aprox. 1 sec.)
Some coalescing parameters are not used or have
different defaults on some chips
MSI : Enabled (if supported by the chip and passed
the interrupt test)
TSO : Enabled on newer chips that support TCP segmentation
offload in hardware.
Unloading and Removing Driver
=============================
To unload the driver, use ifconfig to bring down all eth# interfaces opened
by the driver, then do the following:
rmmod tg3
Note that on 2.6 kernels, it is not necessary to bring down the eth#
interfaces before unloading the driver module.
If the driver was installed using rpm, do the following to remove it:
rpm -e tg3
If the driver was installed using make install from the tar file, the driver
tg3.o (or tg3.ko) has to be manually deleted from the system. Refer
to the section “Installing Source RPM Package” for the location of the
installed driver.
Driver Messages
===============
The following are the most common sample messages that may be logged in the file
/var/log/messages. Use dmesg -n <level> to control the level at which messages
will appear on the console. Most systems are set to level 6 by default. To see
all messages, set the level higher.
Driver signon:
————-
tg3.c:v3.53c (Mar 13, 2006)
NIC detected:
————
eth0: Tigon3 [partno(BCM95704CA40) rev 2002 PHY(5704)] (PCI:66MHz:64-bit) 10/100/1000BaseT Ethernet 00:10:18:04:3e:64
eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[1]
eth0: dma_rwctrl[763f0000] dma_mask[64-bit]
Link up and speed indication:
—————————-
tg3: eth0: Link is up at 1000 Mbps, full duplex.
tg3: eth0: Flow control is on for TX and on for RX.
Link down indication:
——————–
tg3: eth0: Link is down.
Posted by aionman on Oct 7, 2010 in
Linux,
Ubuntu Pre-requisites.
I don’t know how much of it its really needed, but I install the whole Sun’s Java 6 packages,
$ sudo apt-get install sun-java6-jdk sun-java-6-jre sun-java6-pluging
sun-java6-source sun-java6-bin
For sure you’ll need the jdk and plugin one… If your internet connection is slow you might try installing these two first and try the above command.
Configuration of Java for Mozilla.
For enable Java plugin on Firefox 3.6,
$ sudo update-alternatives --install /usr/lib/mozilla/plugins/libjavaplugin.so \
mozilla-javaplugin.so /usr/lib/jvm/java-6-sun/jre/lib/i386/libnpjp2.so 50
Now it’s time to restart your browser.
For checking that the plugin is enabled, open your Firefox browser and type
about:plugins
in the address line, there must be the Java application enabled.
Posted by aionman on Apr 28, 2010 in
Linux,
Ubuntu 1. Download ionCube loaders
sudo wget http://downloads.ioncube.com/loader_downloads/ioncube_loaders_lin_x86.tar.gz
2. Extract
sudo tar zxvf ioncube_loaders_lin_x86.tar.gz
3. Move to a permanent location
sudo mv ioncube /usr/local/
4. Add reference to your php.ini file (sudo pico /etc/php5/apache2/php.ini)
zend_extension = /usr/local/ioncube/ioncube_loader_lin_5.2.so
There are a few versions of the loader in the tar archive. Use the one that matches your PHP version.
5. Restart apache
sudo /etc/init.d/apache2 restart
6. check installation using following command:
php -v
you should see following output:
PHP 5.1.6 (cli) (built: Apr 7 2009 08:00:04)
Copyright (c) 1997-2006 The PHP Group
Zend Engine v2.1.0, Copyright (c) 1998-2006 Zend Technologies
with the ionCube PHP Loader v3.1.34, Copyright (c) 2002-2009, by ionCube Ltd.
Posted by aionman on Apr 11, 2010 in
Linux,
VMWare I think I figured it out. It isn’t related to SLlinux, nor to port 443, nor to VMWare and the FC5 port directly, but a combination of the Fedora port and VMware settings. It is related to the MTU setting consistancy between the virtual machine, FC5 and the site in question. For some reason the FC5 was set to 1492 and not 1500 as set under FC3. For this reason some of the HTTPS sites, I assume, require 1500 mtu. All I did was set Fedora’s eth0′s MTU setting to 1500 (ip link set eth0 mtu 1500) and it works fine. Weird. I could be wrong, but I think that was the problem. At least now it works. I got this hint from the guys at Fidelity. They thought it was unrelated, but apparently it is.
Maximum Transmission Unit(MTU), the largest physical packet size, measured in bytes, that a network can transmit. Any messages larger than the MTU are divided into smaller packets before being sent.By optimizing the MTU setting you can gain substantial network performance
increases, especially when using dial-up modem connections.
Default MTU Size for Different Network Topology
| Network |
MTU(Bytes) |
| 16 Mbit/Sec Token Ring |
17914 |
| 4 Mbits/Sec Token Ring |
4464 |
| FDDI |
4352 |
| Ethernet |
1500 |
| IEEE 802.3/802.2 |
1492 |
| X.25 |
576 |
To change the MTU of an interface on GNU/Linux, you just need to use ifconfig command to do so, like this for example
sudo ifconfig eth0 mtu 1492
To change it permanently on Debian, put it in the /etc/network/interfaces file .where almost all network parameters are found. To do this, just add a line mtu to the definition of your interface and save the file.
sudo gedit /etc/network/interfaces
Example
iface eth0 inet static
address 192.168.0.1
network 192.168.0.0
gateway 192.168.0.254
netmask 255.255.255.0
mtu 1492
Daiup Users
For dialup users: the Maximum Transmission Unit (MTU) value can be changed within the file
/etc/ppp/options
Posted by aionman on Nov 12, 2009 in
Linux,
Ubuntu
Replacing A Failed Hard Drive In A Software RAID1 Array
Version 1.0
Author: Falko Timme <ft [at] falkotimme [dot] com>
Last edited 01/21/2007
This guide shows how to remove a failed hard drive from a Linux RAID1/5 array (software RAID), and how to add a new hard disk to the RAID1/5 array without losing data.
Take RAID1 as an example.
1 Preliminary Note
I have two hard drives, /dev/sda and /dev/sdb, with the partitions /dev/sda1 and /dev/sda2 as well as /dev/sdb1 and /dev/sdb2.
/dev/sda1 and /dev/sdb1 make up the RAID1 array /dev/md0.
/dev/sda2 and /dev/sdb2 make up the RAID1 array /dev/md1.
/dev/sda1 + /dev/sdb1 = /dev/md0
/dev/sda2 + /dev/sdb2 = /dev/md1
/dev/sdb has failed, and we want to replace it.
2 How Do I Tell If A Hard Disk Has Failed?
If a disk has failed, you will probably find a lot of error messages in the log files, e.g. /var/log/messages or /var/log/syslog.
You can also run
cat /proc/mdstat
and instead of the string [UU] you will see [U_] if you have a degraded RAID1 array.
3 Removing The Failed Disk
To remove /dev/sdb, we will mark /dev/sdb1 and /dev/sdb2 as failed and remove them from their respective RAID arrays (/dev/md0 and /dev/md1).
First we mark /dev/sdb1 as failed:
mdadm –manage /dev/md0 –fail /dev/sdb1
The output of
cat /proc/mdstat
should look like this:
server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0] sdb1[2](F)
24418688 blocks [2/1] [U_]
md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/2] [UU]
unused devices: <none>
Then we remove /dev/sdb1 from /dev/md0:
mdadm –manage /dev/md0 –remove /dev/sdb1
The output should be like this:
server1:~# mdadm –manage /dev/md0 –remove /dev/sdb1
mdadm: hot removed /dev/sdb1
And
cat /proc/mdstat
should show this:
server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0]
24418688 blocks [2/1] [U_]
md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/2] [UU]
unused devices: <none>
Now we do the same steps again for /dev/sdb2 (which is part of /dev/md1):
mdadm –manage /dev/md1 –fail /dev/sdb2
cat /proc/mdstat
server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0]
24418688 blocks [2/1] [U_]
md1 : active raid1 sda2[0] sdb2[2](F)
24418688 blocks [2/1] [U_]
unused devices: <none>
mdadm –manage /dev/md1 –remove /dev/sdb2
server1:~# mdadm –manage /dev/md1 –remove /dev/sdb2
mdadm: hot removed /dev/sdb2
cat /proc/mdstat
server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0]
24418688 blocks [2/1] [U_]
md1 : active raid1 sda2[0]
24418688 blocks [2/1] [U_]
unused devices: <none>
Then power down the system:
shutdown -h now
and replace the old /dev/sdb hard drive with a new one (it must have at least the same size as the old one – if it’s only a few MB smaller than the old one then rebuilding the arrays will fail).
4 Adding The New Hard Disk
After you have changed the hard disk /dev/sdb, boot the system.
The first thing we must do now is to create the exact same partitioning as on /dev/sda. We can do this with one simple command:
sfdisk -d /dev/sda | sfdisk /dev/sdb
You can run
fdisk -l
to check if both hard drives have the same partitioning now.
Next we add /dev/sdb1 to /dev/md0 and /dev/sdb2 to /dev/md1:
mdadm –manage /dev/md0 –add /dev/sdb1
server1:~# mdadm –manage /dev/md0 –add /dev/sdb1
mdadm: re-added /dev/sdb1
mdadm –manage /dev/md1 –add /dev/sdb2
server1:~# mdadm –manage /dev/md1 –add /dev/sdb2
mdadm: re-added /dev/sdb2
Now both arays (/dev/md0 and /dev/md1) will be synchronized. Run
cat /proc/mdstat
to see when it’s finished.
During the synchronization the output will look like this:
server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0] sdb1[1]
24418688 blocks [2/1] [U_]
[=>...................] recovery = 9.9% (2423168/24418688) finish=2.8min speed=127535K/sec
md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/1] [U_]
[=>...................] recovery = 6.4% (1572096/24418688) finish=1.9min speed=196512K/sec
unused devices: <none>
When the synchronization is finished, the output will look like this:
server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0] sdb1[1]
24418688 blocks [2/2] [UU]
md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/2] [UU]
unused devices: <none>
That’s it, you have successfully replaced /dev/sdb!
Posted by aionman on Aug 14, 2009 in
CentOS,
Linux How To Install VMware Server On A CentOS 5.0 Desktop
This tutorial provides step-by-step instructions on how to install VMware Server on a CentOS 5.0 desktop system. With VMware Server you can create and run guest operating systems (“virtual machines”) such as Linux, Windows, FreeBSD, etc. under a host operating system. This has the benefit that you can run multiple operating systems on the same hardware which saves a lot of money, and you can move virtual machines from one VMware Server to the next one (or to a system that has the VMware Player which is also free).
Also, with VMware Server you can let your old Windows desktop (that you previously converted into a VMware virtual machine with VMware Converter, as described in this tutorial:http://www.howtoforge.com/vmware_converter_windows_linux) run under your CentOS desktop. This can be useful if you depend on some applications that exist for Windows only, or if you want to switch to Linux slowly.
I want to say first that this is not the only way of setting up such a system. There are many ways of achieving this goal but this is the way I take. I do not issue any guarantee that this will work for you!
1 Find Out Your Kernel Version
Before we go on and install additional software, it’s a good idea to find out about your kernel version because in chapter 2 we will install the package kernel-devel which is needed by VMware Server. There are multiple kernel-devel packages available, and to select the right one you need to know your kernel version.
To find out about your kernel version, open a terminal (Applications > Accessories > Terminal):
Then become root by running:
su
Then run
uname -r
The output should look like this:
[root@localhost Desktop]# uname -r
2.6.18-8.1.3.el5
which means you have kernel 2.6.18-8.1.3.el5 installled.
2 Installing Required Packages
Before we install VMware Server, we must install some prerequisites. To install them, go to Applications > Add/Remove Software:
Type in the root password:
The Package Manager opens. Go to the Browse tab and select:
- Development > Development Libraries
- Development > Development Tools
Then go to the Search tab and search for xinetd. Select the xinetd package for installation:
Do the same for the kernel-devel package. Please make sure you select the kernel-devel package that corresponds to your current kernel (so if you kernel is 2.6.18-8.1.3.el5, select the kernel-devel – 2.6.18-8.1.3.el5.i686 package).
Click on Apply afterwards.
The Package Manager will then resolve all dependencies, download the packages, maybe ask you to accept some unknown software keys (please accept them), and finally install the packages.
3 VMware Server
To download VMware Server, go to http://www.vmware.com/download/server/ and click on Download Now:
Accept the license agreement by clicking on Yes:
Then download the VMware Server for Linux .tar.gz file (not the rpm file!) to your desktop (e.g. to /home/falko/Desktop):
To get the serial number you need to run VMware Server, go to http://register.vmware.com/content/registration.html. Fill in your personal details. Afterwards you will get a page with a serial number for VMware Server. Write it down or print it out:
To install VMware Server, open a terminal (Applications > Accessories > Terminal) and become root:
su
Then go to the location where you saved the VMware Server .tar.gz file, e.g. /home/falko/Desktop (replace falko with your own username!):
cd /home/falko/Desktop
Unpack the VMware Server .tar.gz file and run the installer:
tar xvfz VMware-server-*.tar.gz
cd vmware-server-distrib
./vmware-install.pl
The installer will ask you a lot of questions. You can always accept the default values simply by hitting <ENTER>. When it asks you
In which directory do you want to keep your virtual machine files?
[/var/lib/vmware/Virtual Machines]
you can accept the default value or specify a different location where you have more free disk space, e.g. like /home/falko/virtual_machines, but this is up to you and not necessary.
At the end of the installation, you will be asked to enter a serial number:
Please enter your 20-character serial number.
Type XXXXX-XXXXX-XXXXX-XXXXX or ‘Enter’ to cancel:
Fill in your serial number for VMware Server.
After the successful installation, you can delete the VMware Server download file and the installation directory:
cd ../
rm -f VMware-server*
rm -fr vmware-server-distrib/
You will now find VMware Server under Applications > System Tools:
When you start it, select Local host:
With VMware 2.0 The most noticeable change is that the vmware server console is … gone.
Afterwards, you can create virtual machines (or import your virtual Windows machine that you created with VMware Converter):
Posted by aionman on Aug 4, 2009 in
Linux,
Ubuntu This guide shows how to remove a failed hard drive from a Linux RAID1 array (software RAID), and how to add a new hard disk to the RAID1 array without losing data.
I do not issue any guarantee that this will work for you!
1 Preliminary Note
In this example I have two hard drives, /dev/sda and /dev/sdb, with the partitions /dev/sda1 and /dev/sda2 as well as /dev/sdb1 and /dev/sdb2.
/dev/sda1 and /dev/sdb1 make up the RAID1 array /dev/md0.
/dev/sda2 and /dev/sdb2 make up the RAID1 array /dev/md1.
/dev/sda1 + /dev/sdb1 = /dev/md0
/dev/sda2 + /dev/sdb2 = /dev/md1
/dev/sdb has failed, and we want to replace it.
2 How Do I Tell If A Hard Disk Has Failed?
If a disk has failed, you will probably find a lot of error messages in the log files, e.g. /var/log/messages or /var/log/syslog.
You can also run
cat /proc/mdstat
and instead of the string [UU] you will see [U_] if you have a degraded RAID1 array.
3 Removing The Failed Disk
To remove /dev/sdb, we will mark /dev/sdb1 and /dev/sdb2 as failed and remove them from their respective RAID arrays (/dev/md0 and /dev/md1).
First we mark /dev/sdb1 as failed:
mdadm –manage /dev/md0 –fail /dev/sdb1
The output of
cat /proc/mdstat
should look like this:
server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0] sdb1[2](F)
24418688 blocks [2/1] [U_]
md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/2] [UU]
unused devices: <none>
Then we remove /dev/sdb1 from /dev/md0:
mdadm –manage /dev/md0 –remove /dev/sdb1
The output should be like this:
server1:~# mdadm –manage /dev/md0 –remove /dev/sdb1
mdadm: hot removed /dev/sdb1
And
cat /proc/mdstat
should show this:
server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0]
24418688 blocks [2/1] [U_]
md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/2] [UU]
unused devices: <none>
Now we do the same steps again for /dev/sdb2 (which is part of /dev/md1):
mdadm –manage /dev/md1 –fail /dev/sdb2
cat /proc/mdstat
server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0]
24418688 blocks [2/1] [U_]
md1 : active raid1 sda2[0] sdb2[2](F)
24418688 blocks [2/1] [U_]
unused devices: <none>
mdadm –manage /dev/md1 –remove /dev/sdb2
server1:~# mdadm –manage /dev/md1 –remove /dev/sdb2
mdadm: hot removed /dev/sdb2
cat /proc/mdstat
server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0]
24418688 blocks [2/1] [U_]
md1 : active raid1 sda2[0]
24418688 blocks [2/1] [U_]
unused devices: <none>
Then power down the system:
shutdown -h now
and replace the old /dev/sdb hard drive with a new one (it must have at least the same size as the old one – if it’s only a few MB smaller than the old one then rebuilding the arrays will fail).
4 Adding The New Hard Disk
After you have changed the hard disk /dev/sdb, boot the system.
The first thing we must do now is to create the exact same partitioning as on /dev/sda. We can do this with one simple command:
sfdisk -d /dev/sda | sfdisk /dev/sdb
You can run
fdisk -l
to check if both hard drives have the same partitioning now.
Next we add /dev/sdb1 to /dev/md0 and /dev/sdb2 to /dev/md1:
mdadm –manage /dev/md0 –add /dev/sdb1
server1:~# mdadm –manage /dev/md0 –add /dev/sdb1
mdadm: re-added /dev/sdb1
mdadm –manage /dev/md1 –add /dev/sdb2
server1:~# mdadm –manage /dev/md1 –add /dev/sdb2
mdadm: re-added /dev/sdb2
Now both arays (/dev/md0 and /dev/md1) will be synchronized. Run
cat /proc/mdstat
to see when it’s finished.
During the synchronization the output will look like this:
server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0] sdb1[1]
24418688 blocks [2/1] [U_]
[=>...................] recovery = 9.9% (2423168/24418688) finish=2.8min speed=127535K/sec
md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/1] [U_]
[=>...................] recovery = 6.4% (1572096/24418688) finish=1.9min speed=196512K/sec
unused devices: <none>
When the synchronization is finished, the output will look like this:
server1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid5] [raid4] [raid6] [raid10]
md0 : active raid1 sda1[0] sdb1[1]
24418688 blocks [2/2] [UU]
md1 : active raid1 sda2[0] sdb2[1]
24418688 blocks [2/2] [UU]
unused devices: <none>
That’s it, you have successfully replaced /dev/sdb!
Tags: RAID, Rebuild RAID
