Linux

Articles about the Linux operating system.

Quick and easy network bandwidth benchmarking on Linux and MacOS X

A couple years ago, I setup my first gigabit Ethernet network. I wanted to test just how fast it could go with the equipment I gave it (that is, the NICs, cabling, and switches it operated on). Gigabit Ethernet, theoretically, can operate at 1000 Mbit/sec. This translates to 125 MiB/sec, units your OS typically displays when doing downloads (1000/8). How close is your network setup to that maximum? Copying files between PCs, while being a very "real world" test, will be limited by how fast your disks can read or write. A specialized tool is needed.

While many system benchmark suites include network testing tools, most are not easily separate from their suites, and are not easy to install and use.

Enter NetStrain. It's a very simple C application for Linux and MacOS X designed to stress network connections. Unfortunately, it's not included in most Linux distributions or MacOS X, so you need to download and compile it yourself.

After compiling, use is simple. One machine acts as a server, and another machine acts as a client. Start the server first with:

netstraind -4 9999

This starts a server using IPv4 networking on port 9999 (use a different port if you know this is in use; remember to pick one above 1024 if you're not running as root). On your client machine, start the client connect to the server (assumed to be running on IP 192.168.1.2 and port 9999):

netstrain -4 192.168.1.2 9999 send

NetStrain will then try to send as much over your network connection as it can as long as the client is running. NetStrain is very spartan, so there are not a lot of options. In addition to sending, you may want to test receiving, as well simultaneously sending and receiving. Check NetStrain's README for details.

Most likely, you will not get anything near 125 MiB/sec---but hopefully, you'll get better speeds than a normal 100 Mbit connection to make everything worthwhile.

What if you want to make things faster (without buying newer, better hardware)? There are many parameters you can tune on your operating system's networking stack. However, in most modern operating systems, most of them are already set, or are automatically configured (e.g. TCP window scaling). The one major tunable is something called MTU (Maximum Transmission Unit).

Data is transferred over Ethernet in packets; the MTU defines the size of those packets. A larger packet size means fewer packets are needed to send the same amount of data, reducing the amount of processing that needs to be done by your computer, switches, and routers. Your computer's NIC, switches, and routers need to support large-size MTUs, a feature often advertised as "Ethernet jumbo frames." Jeff Atwood wrote an article on the promise and perils of jumbo frames that you may want to read if you're interested.

Adobe releases pre-release Flash 10 for 64-bit Linux

Today, Adobe released 64-bit Flash for Linux. Finally, I can waste time watching ugly, pixelated Internet video on my 64-bit Linux desktop and laptop, just like all of my 32-bit-confined brothers and sisters on the Internet! (Yes, I know about npviewer---let's not go there.)

What's really interesting is that this is Adobe's first 64-bit release of Flash. That is, Linux users got it first, before users of Windows Vista x64 and and MacOS X. It probably does not mean anything, especially since Adobe has mentioned 64-bit flash will be released at the same time across platforms, but you can't help but feel good inside.

Go download it now and remember to report good bugs.

Update: Some quick notes...

  • The tarball provided on the labs website is not the conventional Adobe Flash installer--it just contains the plugin. To use the plugin, drop the .so file into your ~/.mozilla/plugins/ directory.
  • Make sure to uninstall your npviewer-powered 32-bit Flash completely (disabling the plugin within Firefox is not enough). I personally uninstalled it from my system to prevent any conflict.

Creating your own personal aspell dictionary

Something that has bothered me forever is that applications that use GNU aspell for spell checking kept marking my name as a misspelling (I'm looking at you, KMail). Most front-end applications don't provide a way for you to add your own custom words.

Apparently, creating your own personal dictionary is ridiculous easy with aspell.

If your language is English, create a file in your home directory called ".aspell.en.pws":

personal_ws-1.1 en 0
Samat
quasirhombicosidodecahedron

The first line is a required header. Every subsequent line is a word you want to add to your dictionary. I can't believe I've let this sit for so long. Because it's a nice text file, syncing this file between machines to take your dictionary with you is trivially easy.

Taking Drupal sites offline via mysql and the command line

Drupal-powered websites can be put into an "offline mode." This is much better than most alternatives (such as taking the web server offline), especially for search engines, as the message and HTTP status codes given to users and robots alike will tell them to patiently come back later.

I've found that putting the site into offline mode makes database backups go much faster on heavily trafficked sites (which is obvious). However, for a particular site I was working with, this needed to be done in an automated manner, and on a dedicated database server that did not have access to the Drupal installation.

Most people take their Drupal sites offline through Drupal's web-based administration interface. They can also be put offline through the Drupal Shell. Neither were suitable for me: the former cannot be automated easily, and the latter requires access to the Drupal installation. Fortunately, Drupal sites can easily be taken offline by setting things in the database, which can easily be done via bash scripts and the command-line MySQL client.

Given your database user is my_db_user, password my_password, and database my_drupal_db, the backup script would look something similar to:

#!/bin/bash

# Take site offline
mysql --user my_db_user --password=my_password my_drupal_db << EOF
UPDATE variable SET value='s:1:"1";' WHERE name = 'site_offline';
DELETE FROM cache WHERE CID = 'variables';
EOF

# Do stuff here while the site is offline (e.g. backup)

# Bring site online
mysql --user my_db_user --password=my_password my_drupal_db << EOF
UPDATE variable SET value='s:1:"0";' WHERE name = 'site_offline';
DELETE FROM cache WHERE CID = 'variables';
EOF

Update: The original version of this article had some problems on some setups with the variables table being cached. I added another SQL statement to make sure this cache is flushed so the site actually reflects its configuration.

Update: This method really doesn't work that well, and the more I think about it, there isn't a way to get around writing something that interacts with Drupal. I'm working on a script that will be more fool-proof.

Amarok 2 uses MySQL embedded as a metadata store

There's been a bit of turmoil in the Amarok and KDE communities the past week with Amarok's decision to only support MySQL Embedded in Amarok 2. Jeff Mitchell has written about the Amarok design decisions made.

I'm a little bothered by this, as it forgeos all the "semantic desktop" work that has gone into KDE 4, namely what's provided by the Strigi and Nepomuk libraries. One thing the whole semantic desktop concept entails is that other applications will be able use data another application stored, but without care to what that other application was or how it was stored. For example, I should be able to share the list of all tracks in my music library, how many times I've played tracks, what tracks I think are my favorite, etc across music players. This kind of abstraction is, obviously, good for users, but bad for developers of proprietary software. They don't want you to easily switch between applications that they do not control. Amarok switching to it's own database store is a stab at this kind of desktop interoperability. I've my own thoughts to add, though, that support what the developers are doing...

Amarok is an awesome application. Dare I say, it's a killer application on Linux---on several occasions this past year I've recommended people install Linux just so that they could play with Amarok and see how much better it is compared to what they were using (yes, I'm looking at you, iTunes).

Before Amarok, I used Music Player Daemon (mpd). I stopped using it after a while: the playlist management wasn't very good; it would eat those playlists that I spent a lot of effort to make; the GUIs available at the time were lacking; and it was very slow when working with tens of thousands of songs. Some of this may have changed but I've not been motivated to look back.

Enter Amarok: I switched because the playlist management was so much better. I setup a MySQL server on my workstation to store metadata, as SQLite was much too slow. Amarok backed with MySQL is very fast---I dare others to find a library-based music manager that is faster with the number of songs I've thrown at it.

Balancing desktop interoperability with performance is a delicate balancing act. Interoperability is the hot thing these days---look at how Apple's line of integrated software and hardware continue to sip market share from the Microsoft-powered desktop. But when it comes down to it, performance and other more perceived benefits are going to win out over desktop interoperability. The Amarok developers' decision to go with MySQL embedded is a good one that will hopefully keep people moving to Amarok over proprietary alternatives.

Speeding up SpamAssassin rule processing on Debian and Ubuntu

SpamAssassin is one of the most-used spam filtering systems in use today. Unfortunately, because there are so many different ways SpamAssassin can be used, SpamAssassin remains subject to many performance problems. Fortunately, there are several speed-ups and optimizations that can be applied to most SpamAssassin installations to speed up its rule processing, especially on Debian and Ubuntu GNU/Linux-based systems. These instructions can be adopted to other operating systems as well.

This article does not discuss configuring your mail filtering system (i.e. procmail, maildrop). This depends completely on your setup, and more than likely there are plenty of other articles that describe the best way to setup what you want.

The number one thing Dell customers want... Linux

Dell Ideastorm is a Dell-run website where users can post and vote (anonymously) for ideas for Dell products. What's the number one thing people want...?

People want Linux on their Dell machines. Remember to click the "promote" icon and make your vote count.

Syndicate content