Linux resources

 
Find and bind key sequences in bash
 
Learn how to use keyboard function keys within bash, or any other shell, by binding the key sequences to a particular shell command.
Most keyboards today come with an extra row of function keys at the top of the keyboard. These function keys can be customized and used within bash, or any other shell, by binding the key sequences to a command in the shell. Some keys may be intercepted by the window manager or the terminal program, such as konsole or gnome-terminal. You can retain those key bindings and use unassigned keys inside the shell, or you can reconfigured them to use a certain key in the shell instead.
To obtain the key sequence from a function key, use the read command. The following is an example of pressing the [F12] key:
$ read
^[[24~
Note that different keyboards will produce different key sequences, and modifiers to the function keys (such as [Ctrl][F12] or [Shift][F12]) will produce other sequences as well.
The next step is to bind that key sequence to a particular shell command. For example, you can bind [F12] to the "history-search-backward" shell command:
$ bind '"\e[24~": history-search-backward'
Make sure you write the key sequence as \e[24~ rather than ^[[24~. This is because the ^[ sequence is equivalent to the [Esc] key, which is represented by \e in the shell. So, for instance, if the key sequence was ^[[OP the resulting bind code to use would be \e[OP.
Not only does the bind command bind function keys, but you can also use bind to map key sequences (such as [Esc][P] or [Esc][Q]) by writing the bind key code as \ep and \eq respectively.
For a list of shell commands that you can use, examine the /etc/inputrc file.
 
 
Apache
Analyze Apache logs with Analog
If you're looking for a useful log analysis program, check out Analog. This powerful, fast tool creates Web pages based on the analysis of Apache log files.
If your Linux vendor doesn't provide binary packages, you may have to download and install the program from source. After installation, create a configuration file that tells Analog what logs to read and where to place the output.
If installed via RPM or DEB, Analog will typically place a default configuration file in /etc/analog.cfg. Make a copy of this file, and customize it to fit your needs. Here are the essentials you need to set:
LOGFILE /var/log/httpd/access_log
HOSTNAME www.myhost.com
HOSTURL http://www.myhost.com
OUTFILE /var/www/html/logs/report.html
CHARTDIR /logs/images
LOCALCHARTDIR /var/www/html/logs/images
This tells Analog which log file to analyze, provides information on the host it's analyzing (i.e., hostname and URL), and indicates where to place the report file. (In this case, the resulting URL would be http://www.mysite.com/logs/report.html.) It also tells Analog where to write the image files for the charts that it creates.
Analog creates a very comprehensive output that includes a number of statistics, such as monthly page views, daily and hourly summaries of page requests, most used search requests to reach the site, and more.
For an up-to-date report, run Analog every day by using the following:
# analog -G +g/etc/myanalog.cfg
This assumes your customized configuration file is /etc/myanalog.cfg, and it tells Analog to use the specified configuration file instead of the default configuration file. This comes in handy if you've configured Apache to create log files for different virtual hosts and want a different report for each virtual host.

Storage Hardware and Terminology

iSCSI. Fibre Channel switch. TOE NIC. When do I need an "HBA" for storage and when can I just use a standard Ethernet adapter? In the world of storage, storage terms and concepts can serve to further muddy waters that are already somewhat unclear to many IT pros. After all, up until fairly recently, a storage area network was the purview of only large organizations with large IT budgets and specialized staff. Now, however, as more storage options become available to small- and medium-size businesses, discussion about this topic is starting to take place in these IT departments. In this article, I'll go over some of the terms and equipment related to different types of storage solutions to help you get a better understanding of these increasingly cost-effective ways to handle your storage needs.

Network Attached Storage

NAS systems are ones that connect directly to your network, but that do not generally provide block level communication with the host, making them unsuitable for most database and Exchange applications. A NAS system is really just a mondo file server running its own operating system and providing direct access to users. That is, users can directly access the files on the NAS device just like they access files on a file server. Each whole file is transferred between the NAS device and the requesting client.

NAS terminology

• NAS head: This is the part of the NAS to which clients connect. Behind the NAS head may lay hundreds or thousands of gigabytes of available storage, but clients need to access this space via the NAS head.
• NFS: NFS (Network File System) is one of the communications protocols usually supported by NAS heads for communication with network clients, particular those of the UNIX or Linux flavor, although NFS clients are available for just about any operating systems these days.
• CIFS (the protocol formerly known as SMB [Server Message Block]): CIFS (Common Internet File System), the protocol primarily responsible for file sharing communication with Windows (and Linux-based Samba) servers is another commonly supported protocol in most NAS heads. CIFS/SMB is used for communication with the NAS head by most Windows clients. Both NFS and CIFS use TCP/IP for their underlying communication.

Hardware and software needed to support NAS systems

One beauty of NAS systems is simplicity. If you have an existing Ethernet (Fast or Gigabit are the best choices here) network—and who doesn’t these days?—you can almost literally just pop a NAS head and storage on your network and be on your way. In short, the only equipment you need to support a NAS system in your environment is an Ethernet connection to the NAS head. For additional reliability, you might want to configure your NAS hardware with multiple connections, but at the end of the day, just typical Ethernet switch ports are all you need for a NAS system to work.

On the software side, you might need an NFS client on your Windows computers, or an SMB client (such as Samba) on your Linux computers to access the NAS system. However, this is only true if you're trying to access a NAS device that does not include support for your client operating system.

iSCSI-based Storage Area Networks

Compared to NAS and fibre channel-based SANs, iSCSI is the relative new kid on the block in the storage world, but due in no small part to its very low cost, has started to give fibre channel a serious run for its money. iSCSI storage networks are a complete technology—ranging from iSCSI drivers on your servers to storage hardware based on iSCSI standards. Unlike NAS systems, iSCSI SANs are perfect for database and Exchange applications due to the fact that iSCSI transmits block level data rather than complete files.

"Block level?," you might ask—and it's a really good question, particularly since storage vendors throw the term around pretty loosely. Block level communication means that data is transferred between the host and the client in chunks called blocks. Databases and Exchange servers depend on this type of communication (as opposed to the file level communication used by most NAS systems) in order to work properly. That said, some NAS vendors' devices are certified for use by databases and Exchange, but I don't generally recommend this configuration unless it's a last resort.

iSCSI terminology

Being somewhat new, the introduction of iSCSI to the marketplace has been accompanied by some new terminology.

• iSCSI: It might be nice to talk a little about the term iSCSI itself. You probably know about SCSI (Small Computer Systems Interface) and probably even use it on most of your servers. SCSI has long been known for its reliability and speed. iSCSI is simply the commands used by SCSI systems encapsulated inside TCP/IP – hence the 'i' in iSCSI. Put another way in some iSCSI documentation, iSCSI is a way for a storage initiator such as a server to send commands to a storage target such as array of disks. iSCSI "targets" don't even have to use SCSI disks. In fact, many of them use newer serial ATA (SATA) disks and translate the SCSI commands for use on these less expensive devices.
• iSCSI driver: An iSCSI driver attaches to a standard Ethernet adapter (usually of the gigabit variety) and facilitates communication with an iSCSI storage array. What does this mean for you? Most importantly, it means that you can start using the features of storage area network without having to buy expensive, specialized adapters for your servers. As long as your server has a gigabit Ethernet adapter, you can use iSCSI. If you do decide to use just an iSCSI driver (a lot of people who use iSCSI do use these), I recommend using a second gigabit Ethernet adapter in your server and create a separate network for storage communication. With today's overpowered dual- and quad-processor servers, this type of communication is almost always sufficient and you don't need to worry about TOE NICs (below).
• TOE (TCP Offload Engine) NIC: For servers that are under a very heavy load, the additional load required to encapsulate commands destined for the iSCSI target can be a killer. By some estimates, depending on what you're doing, you might eat up to 30% of your CPU with iSCSI overhead, although this is not very common. For instances in which this level of overhead is unacceptable, you can offload the work to a specialized NIC called a TOE NIC. As the name implies, a TOE NIC handles the encapsulation, thus freeing up the CPU for other tasks. I recommend serious testing before you invest in TOE NICs. Measure your server's CPU to see what amount of processing is dedicated to the encapsulation task for iSCSI.
• iSCSI initiator software: This is software that either comes with the host operating system and binds to a standard Ethernet NIC, or that resides on an iSCSI TOE adapter. The iSCSI initiator software is responsible for processing iSCSI commands and for managing the TCP/IP communications with an iSCSI storage array. Most modern operating systems include iSCSI initiator software at no additional charge. In a software-only scenario using the OS's iSCSI drivers and a standard Ethernet NIC, the host processor is responsible for translating iSCSI commands. These initiators work with just about any gigabit server NIC and are generally more than adequate with respect to performance. For older, slower servers, you might want to consider a hardware-based initiator such as a TOE NIC.
• iSCSI target: This can be any device with which your host communicates using iSCSI, including an iSCSI disk array or iSCSI-aware tape unit.

Hardware and software needed to support iSCSI systems

I've already gone over most of the items you need to support an iSCSI infrastructure. The really great part about iSCSI is that implementation is fairly inexpensive since you probably have everything you need, except the disks. On the server side, new versions of Windows, Linux, UNIX, and NetWare all include iSCSI initiators, and with today's really fast servers, iSCSI overhead using a standard NIC is negligible on all but the most loaded servers. Beyond the host, to interconnect your servers and storage devices, all you need is a standard gigabit Ethernet switch on a network separate from your client communications. I recommend a separate network for two reasons: (1) when you separate client traffic and storage traffic, overall storage network performance stays high; and (2) since you probably don't want your clients directly accessing storage except via the server, you can help to secure your storage network by keeping it separate from your primary network. Last, but certainly not least, you need iSCSI targets—namely a storage array—with which to work.

Fibre Channel-based Storage Area Networks

The granddaddy of storage networks, Fibre Channel-based storage remains the strongest player in the networked storage market, although iSCSI has quickly become a formidable competitor. Like iSCSI, FC SANs transfer data at the block level making FC SANs more than suitable for database applications and Exchange rollouts. Also like iSCSI, FC uses its own terminology and introduces some new technology to the IT infrastructure.

FC SANs have long enjoyed their position as the respected top-tier in storage architecture mainly due to their reliability, performance, and ability to protect their data.

Although iSCSI has become a fierce competitor in some markets, the future for FC still looks good with plans to increase speeds from the current standard of 2Gbps to 4Gbps, although the jury is out on how much of an impact this really will have on overall performance due to other limitations in storage.

iSCSI terminology and required hardware

FC-based SANs brought a slew of terminology to the forefront of the minds of storage experts. I'm not going to go over every term here, but will provide you with the ones that are important to know when you're comparing storage solutions. I'm combining the sections on terminology and hardware for this topic since they are both pretty much the same.

• HBA (Host Bus Adapter): An FC HBA is server hardware that enables communication with FC storage hardware. In most cases, the term HBA is coupled with an FC solution, but some vendors also refer to iSCSI TOE NICs as HBA, so just watch what you're reading. An FC HBA is generally an add-on card that utilizes a PCI slot.
• FC switch: FC SANs use specialized equipment, including the switch interconnects between the hosts and the storage. If you're new to the storage game, prepare for a jaw drop when you see the price tags associated with FC switches! They're expensive and usually use GBICs so you can pick the kind of connectors you want to use. You can connect FC switches together to expand your overall storage fabric to support thousands of nodes.
• Node: A node is any device connected to a fibre channel switch, be it a server, a storage system, or a tape drive.
• Adaptec and other suppliers have offloaded the TCP/IP processing by adding dedicated TOEs (TCP/IP Offload Engines) to their Ethernet cards used for storage networks. But these are more expensive than ordinary Ethernet NICs and so the price advantage of iSCSI SANs compared to Fibre Channel SANs is reduced.
• Intel sees an opportunity here and aims to add data set copying to its server chipsets and network controllers via I/O AT. This will be done in parallel to the TCP/IP processing taking place in the server CPU thus speeding up the overall data transmission time. In effect iSCSI commands and data moving are done simultaneously with Intel network controller memory being directly accessed.

BDS Unix BIND

BIND is a DNS server package that's had a rather spotty history when it comes to security. However, despite these limitations, there are few alternatives for serving up DNS data that are as feature-rich as BIND.

If you just need to serve up DNS data without support for zone transfers, keys, and other features that BIND offers, using something like D.J. Bernstein's djbdns package may be sufficient. But if you need some of the more robust features that only BIND offers, you might as well learn a few things you can do to better secure your setup.

First, configure BIND not to report its version number. This can stop passive scanners from identifying the version of BIND you're using.

This trick doesn't really secure BIND as much as it obfuscates things a bit. You can do this by editing the named.conf file, as shown below:

options {
version "Not available";
}

You can also restrict which hosts can perform zone transfers. BIND configurations typically have no restrictions for performing a zone transfer, which can lead to providing unwanted data to potential attackers.

You can also set this restriction using the named.conf file. Here's an example:

options {
allow-transfer { 192.168.5.10; };
}

This restricts zone transfers to 192.168.5.10, which would be your secondary DNS server. You can also use Transaction Signatures (TSIG) to more securely perform zone transfers.

You should also disable recursive queries, which prevents your DNS server from being vulnerable to spoofing attacks. Add the following to the named.conf file:

options {
fetch-glue no;
recursion no;
}

Finally, you may also want to consider running BIND in a chrooted environment as a nonprivileged user. (BIND's documentation discusses how to do this.)

By running BIND in a chroot, you're locking it into a special section of your system where it can't interact with the rest of the system, minimizing the damage potentially caused by an attacker who successfully exploits it.

Solaris 10 Routing

The key thing is in your routeadm output. By default in Solaris 10 IP forwarding and routing are off.

To enable IPv4 forwarding, as you have, you can set
#routeadm -e ipv4-forwarding

To enable routing, use
#routeadm -e ipv4-routing

To make your changes take effect and start the routing daemon, use
#routeadm -u

Use the same commands (with ipv6) to enable the IPv6 routing and forwarding.

These changes persist across a reboot.
This should take care of your problem.

Learn the many uses of netcat

Often referred to as the "Swiss Army Knife of networking," netcat is a tool that administrators can use to read and write TCP or UDP data across the network. In addition, it's extremely useful for network debugging and testing.

Netcat offers several interesting uses. For example, you can make it listen to a particular port and run a program. To do so, use the following:

$ netcat -v -l -p 10111 -e "/bin/cat /etc/motd"

This tells netcat to listen to port 10111. When there's a connection, it tells netcat to execute "/bin/cat /etc/motd," which essentially displays the contents of /etc/motd and exits.

You can also set up netcat on a machine to listen for incoming connections and run it on a remote machine to connect to the local machine and serve up a bash shell. For example, on a local machine with an IP address of 192.168.5.10, you would use the following:

$ netcat -v -l -p 10111

On the remote machine, you would use:

$ netcat 192.168.5.10 10111 -e /bin/bash

This tells the netcat instance on the remote machine to connect to the netcat instance listening on 192.168.5.10 and serve up a bash shell from the remote machine, which will then be available on the local machine. Using the netcat instance on 192.168.5.10, you can execute shell commands on the remote host.

To perform some Web debugging, you could use something like the following:

$ netcat www.website.com 80

Then, enter typical HTTP commands to get the unaltered output (e.g., "GET / HTTP 1.0").

Sun Solaris Command Tips

UNIX backups/Restore

Commands to take backup of /usr /var /lib directories to tape and disk using "tar"

a) tar -cvf /dev/rmt/0 /usr /var /lib [Taking into tape ]
b) tar -cvf backup_file.tar /usr /var /lib [ Taking disk image]

Taking backup to a tape device attached to a remote system
a) tar -cv /var /usr /lib | rsh remote_hostname dd of=/dev/rm/0 obs=128
b) ufsdump 0ucf remote_hostname:/dev/rmt/0 /file_system

Extracting / Restore the contents of a tape /tar file

a) tar -xvf /dev/rmt/0 [ restore contents of a tar tape ]

b) tar -xvf filename.tar [ Restore contents of a tar file ]

c) ufsrestore f /dev/rmt/0 filename [Restore from ufsdump tape backup]
d) ufsrestore rf remote_hostname:/dev/rmt/0 filename [ Restore from remote host tape device ]
List contents of backup
a) tar -tvf /dev/rmt/0 [ List contents of a tar tape ]
b) tar -tvf filename.tar [ List contents of a tar file ]
c) ufsrestore tf /dev/rmt/0n [ list contents of a tape of ufsdump]

Processes and Processor

What are the types of priority classes supported by Solaris ?
Timesharing, system and real-time

How will you place a running process in real time class?
priocntl -s –c RT –I uid process_id_number

How will you start a process in real time class?
priocntl –c RT –e process_name

How will you bind a process to processor ?
pbind –b processor_no process_id

Network

What is the pocket size used by SQL*NET Version 2 on Solaris
Default is 2K

What is the procket size used by network layer TCP/IP in SQL*NET v2?
Default 1K

Inter process communication (IPC) tuning

What is post wait driver ?

The post wait driver reduces the overhead incurred by the more expensive use of

semaphore operation for interprocess communication.

Memory

What is the kernel parameter which controls the UNIX buffer cache on solaris ?
bufhwm ( bufhwm is the maximum amount of physical memory, in kilobytes that can be used byI/O buffers)

While starting an oracle process , the unix system displays an erro message like "Cannot allocate more shared memory segment for the processes" If oracle person approaches you, how will you fix this problem?
ANS: increase the value for kernel memory parameter "set semsys:seminfo_semmns" in /etc/system file. [ this is a blind way]

System configurations

How do you find out total RAM installed on your machine
Ans: $prtconf |grep size

How do you find no. of processors installed on your sun box and processing speed?
Ans: $psrinfo –v

How do you find out how many disks are available on your system?
Ans: $ format (for sysadmins)
$ iostat –E (for users – count only the disks which has disk size correctly)

System Boot Options

How do you boot sun box in single user mode?
ANS: At ok prompt type "boot -s" [ ok boot -s OR ok boot -1 OR ok boot -S ]

What are the commands to shutdown the system?

ANS: shutdown, init , halt, reboot

Devices

What is the meaning of logical name of a disk drive format like c0t0d0s0?

ANS: c0 - Controller number

t0 - SCSI bus target number

d0 - Disk number
s0 - Partition or slice number

List some of the main device types?

ANS: /dev/dsk - Disk devices
/dev/rdsk - Raw or character devices
/dev/rmt - Tape devices
/dev/term - Serial line devices
/dev/pts - Pseudo terminals

List commands to display physical devices ?
ANS: prtconf
Sysdef
Dmesg
sysinfo


Note All commands are specific to Sun Solaris operating system

Journeys of March 2005

This month is very hectic for me almost every week I am traveling to different part of India. This Traveling starts from 17th Feb 2005 onwards. First to and fro to Mumbai where I met and stayed with some of my old friends. Then frequent two visit for Bhopal.In every journey I had an interesting discussions with all of my travel mates. Most of the discussion is for our nation. I write almost all these experiences in my diary earlier but this time I decided to publish all those moments on blogs.

Our journey to Bhopal starts at 9.00 PM dot via “Shan e Bhopal” i.e. Bhopal Express which is among first in India who received ISO 9000. As soon as I enter in my coach on my birth there is one respectable Sardar Ji who itself initiated the intro with me. Later I get to know that he is retired Col and presently serving Nation as director and principle of “ Maharishi Institute of Management” why I am saying serving nation because he is feeding the knowledge hungry youth with his knowledge and experiences. Slowly some other people also joined One gentle man of approx 45 years named Sanjay Bhatnagar interior designer working for some of the big government projects and Mr. Parvej Sharif a property dealer in Bhopal. All of them are double of my age.

In this journey what I concluded is that the nation is really lacking the good leadership. Its my pleasure that on one end I am reading the book “Ignited Minds” On other hand I am able to see the practicality of what all is conveyed by author in this book. All of them are very experienced and very knowledgeable personality. Don’t know why and how the discussion about the Developed India by 2020 is emerged among us. The communication starts with the train; most of them taking about the train about how well its maintained and its sharpness about the timings. (But like to tell you that the train is very sharp and on time if it travels from NZM to BPL while it usually late if u travel from Bhopal to Nizamuddin.) Why I am telling this because in a month I travel almost 4 times in this train to see my parents.

Most of our leaders are of 60+, which is the retirement age. What I analyzed is that although aged people have very good experience but some time this experience is …………. Because on one side they can use this enriched experience on the other side they have very negative approach this negative and lack of enthu and initiative is all because of experiences which they gained in their past. Among all of them I found very few having sprit and enthu to achieve targets including my father. Also they are very easily convinced by each other’s negative approach. But exceptions are everywhere one of such exceptions is Dr. A P J Abdul Kalam” president of India. Country need the enthusiasm to do things better in our leaders. Always aged and experienced people try to find out the impossibilities first as compare to possibilities.

I am not able to participate in their discussion because I am too young to say any thing to them. Most of us are dreaming about the developed nation (where we are living i.e. India). But unfortunately Mr. Sanjay has lots of contradiction about dream, which I am dreaming. What I say above is that they try to find out impossibilities first and so Sanjay too. What I feel is experience should provide us warning so that the mistakes, which we made earlier, should not be repeated again and there should be no glitches to our progress for future.

Most of the people criticize India because of corruption; I am not able to understand corruption comes and nourished by whom; its we who feed corruption. I am enclosing some of the examples of mime which will prove that we ourself only responsible to this. I am traveling to Bhopal to Collect PCC (Police Clearance Certificate) most of my seniors suggested me that I should give some gift (hopefully u understand) at passport office to get the PCC in a day or so. Unfortunately almost 3 to 4 peoples suggested me this and I am also convinced with them. But in train all theses discussion and the book “ Ignited Minds” changed my view hence I decided that I will firstly try to get it done my own because I didn’t see any complication in my case and hope that it could be done quite smoothly. Wonderful I got it without nourishing the corruption.

Another comments that Mr. Sanjay has raised are technology adoption by Indians. Like the mobile usage in India by every individuals including lower segments and the technology usage in the entertainment industry. He also says to me that my child will talk to me on mobile in spite of direct interaction. Reliance approach is to make things cheap and affordable so that every individual has mobile; His (Sanjay) slogan towards the Ambanis "Kar lo duniya mutthi mein" is "Kar do duniya mitti mein" he also seems very angry to technology because his elder son lost his 19 hrs work on the tele film editing because of hard disk crash. That is his view but like to convey him is that there is always two approach to see any thing. I am including my view.
Very recently on 18th March I visited a very ruler area of UP near Allahabad i.e. "Chakghat". 3 yrs ago seen their is like this On landline u will receive a call very rarely even today in some area the situation is same but because of the mobile technology they are also in position to be in touch to their dear one without traveling for 10 KM. In that ruler area people are blessing the Ambanis for making mobile available to even to rishka walas. Although I am leaving far from my parents but still I am in regular touch with my parents due to mobile what’s harm or wrong if I am taking to my parents on mobile.

My parents have their old house in a little ruler area so called “Jasra” near Allahabad. Three yrs earlier there is no way to communicate to my parents if they are going to visit their house but now although reception quality is very poor, and signal reception in our house of 30*110 foot is for only 3 foot on the roof but at least I am able to be talk to them conveniently. That’s true to communicate to me any message they have to reach in that small reception area but still at last able to be in my touch.

He is saying that his son lost his 19 hrs work due to hard disk crash. Is it technology problem? Definitely not! What I feel is that he has done the work of 100 hrs in Just 18 hrs because of new advanced technology. But its unfortunate that we Indians feel that we are masters in all field and without knowing in and out of any thing we feel that we are able to take best decision our own. Basically we Indians are Jack of all and master of none including me. I suggest him that if his son consulted some good system architect then he definitely suggests him to invest Rs6000/- more in his Rs80000/- investment so that the hard disk will be get mirrored and he will not loose his critical and important data due to some hardware failure.