NFSv3 with IPv6 on SLES11 SP2

SLES11 SP2 comes with 3.0.0 and nfs-utils 1.2.3

sles11sp2:~ # uname -a
Linux sles11sp2 3.0.13-0.19-default #1 SMP Fri Feb 3 15:38:23 UTC 2012 (7f256ae) x86_64 x86_64 x86_64 GNU/Linux
sles11sp2:~ #
sles11sp2:~ # rpm -qa | grep nfs-
limal-nfs-server-1.5.3-0.2.11
yast2-nfs-server-2.17.7-1.1.2
nfs-kernel-server-1.2.3-18.17.2
nfs-doc-1.2.3-18.17.2
yast2-nfs-common-2.17.7-1.1.2
nfs-client-1.2.3-18.17.2
limal-nfs-server-perl-1.5.3-0.2.11
yast2-nfs-client-2.17.13-0.5.189
sles11sp2:~ #

Even though the kernel 3.0.0 with nfs-utils-1.2.3 has support for NFSv3 with IPv6 thich seems to be not working with SLES11 SP2

sles11sp2:~ # rpcinfo | grep -w nfs
    100003    2    tcp       0.0.0.0.8.1            nfs        superuser
    100003    3    tcp       0.0.0.0.8.1            nfs        superuser
    100003    2    udp       0.0.0.0.8.1            nfs        superuser
    100003    3    udp       0.0.0.0.8.1            nfs        superuser

This seems to be due to Novell change in the kernel which disabled NFSv3 with IPv6. This was probably part of SLES11-SP1 which was just blindly applied over SLES11SP2

Once you install the kernel sources 'kernel-source-3.0.13-0.19.1', then in /usr/src/linux-3.0.13-0.19/net/sunrpc/svc.c

                        if (strcmp(progp->pg_name, "nfsd") == 0 &&
                            i < 4 &&
                            family == PF_INET6) {
                                /* Don't register NFSv2 or NFSv3 for IPv6
                                 * protocols as we don't support statd
                                 * on IPv6 yet
                                 */
                                dprintk("svc: ... not telling portmap\n");
                                continue;
                        }

This additional check is there on even SLES11-SP1. The nfs-utils (nfs-client and nfs-kernel packages) that are available with SLES11SP1 did not have support for IPv6 in user level daemons like mountd/statd. But with SLES11SP2 and nfs-utils-1.2.3. the support for IPv6 is available in user level daemons so it does not make sense to forcibly disable this. This seems to have been just carried over from SP1 without SP2.

Once you disable this part of code as

#if 0
                        if (strcmp(progp->pg_name, "nfsd") == 0 &&
                            i < 4 &&
                            family == PF_INET6) {
                                /* Don't register NFSv2 or NFSv3 for IPv6
                                 * protocols as we don't support statd
                                 * on IPv6 yet
                                 */
                                dprintk("svc: ... not telling portmap\n");
                                continue;
                        }
#endif

and recompile the module as (you would linux-header, kernel-default-devel and gcc packages installed)

make -C /lib/modules/3.0.13-0.19-default/build M=`pwd` modules

The new module that gets created works well with IPv6 over NFSv3. Once the module has been build by the above make command, place the new module in /lib/modules/2.6.32.43-0.4-default/kernel/net/sunrpc/sunrpc.ko. and reboot the machine to use the new sunrpc module

After reboot if you run 'rpcinfo' you should be able to see that for NFS version 3 IPv6 socket is registered with rpcbind

sles11sp2:~ # rpcinfo | grep -w nfs
    100003    2    tcp       0.0.0.0.8.1            nfs        superuser
    100003    3    tcp       0.0.0.0.8.1            nfs        superuser
    100003    2    udp       0.0.0.0.8.1            nfs        superuser
    100003    3    udp       0.0.0.0.8.1            nfs        superuser
    100003    2    tcp6      ::.8.1                 nfs        superuser
    100003    3    tcp6      ::.8.1                 nfs        superuser
    100003    2    udp6      ::.8.1                 nfs        superuser
    100003    3    udp6      ::.8.1                 nfs        superuser

Setting up LDAP server on suse 10

Start LDAP services using Yast

Start yast from terminal as
# yast


In 'Network Services' -> 'LDAP Server', Start the LDAP server. And Select 'Configure...'.

Create a new database like
* Base DN - dc=test,dc=com
* Set the password for root DN

Save and Exit yast.

Verify the DN has been created using
* ldapsearch -x -b dc=test,dc=com


Download MigrationTools as given in
http://tazlambert.wordpress.com/2008/05/08/ldap-server-openldap-in-opensuse-102/
And set the base as  in migration_common.ph


$DEFAULT_BASE = "dc=test,dc=com" #your base suffix or domain name

Then you can run:

chmod -Rf 770 /MigrationTools-47 
./migrate_base.pl > base.ldif

Edit the base.ldif to remove the first object which specifies the dc=test,dc=com as we have already set that up using yast. Remove the following lines form base.ldif

You have to edit base.ldif so that it will become like this:

dn: dc=test,dc=com
dc: test
objectClass: top
objectClass: domain
# ldapadd -x -W -D "cn=administrator,dc=test,dc=com" -f base.ldif




Create the passwd ldif as
./migrate_passwd.pl /etc/passwd > passwd.ldif
 ldapadd -x -W -D "cn=administrator,dc=test,dc=com" -f passwd.ldif

Group ldif file can be created as
./migrate_group.pl group.in > group.ldif 

ldapadd -x -W -D "cn=administrator,dc=test,dc=com" -f group.ldif

If while adding the group.ldif you receive an error saying
ldap_add: Object class violation (65)
        additional info: no structural object class provided


Then check in /etc/openldap/slapd.conf if rfc2307bis.schema is being used. If that is true then according to http://www.openldap.org/lists/openldap-technical/201004/msg00082.html, rfc2307bis defines posixAccount as AUXILLARY, comment out rfc2307bis.schema and use nis.schema

include         /etc/openldap/schema/nis.schema
 



Only POSIX ACLs supported over NFS

NFS server only supports posix acls, i.e, system.posix_acl_access and system.posix_acl_default. Other extended attributes are not supported through NFS server. There is strict checking in NFS that only these 2 ACLs can be set/get.

2220 int    
2221 nfsd_set_posix_acl(struct svc_fh *fhp, int type, struct posix_acl *acl)
2222 {      
2223         struct inode *inode = fhp->fh_dentry->d_inode;
2224         char *name;
2225         void *value = NULL;
2226         size_t size;
2227         int error;
2228        
2229         if (!IS_POSIXACL(inode) ||
2230             !inode->i_op->setxattr || !inode->i_op->removexattr)
2231                 return -EOPNOTSUPP;
2232         switch(type) {
2233                 case ACL_TYPE_ACCESS:
2234                         name = POSIX_ACL_XATTR_ACCESS;
2235                         break;
2236                 case ACL_TYPE_DEFAULT:
2237                         name = POSIX_ACL_XATTR_DEFAULT;
2238                         break;
2239                 default:
2240                         return -EOPNOTSUPP;
2241         } 
2242        

bash: performance of let vs expr vs (( ))

A simple counting loop is used to test the performance of the bash utilities

# i=1; time while :;do [ $i -gt 1000000 ] && break; let i=$i+1; done; echo $i

real 0m17.590s
user 0m17.109s
sys 0m0.400s
1000001

# i=1; time while :;do [ $i -gt 1000000 ] && break; i=$(($i+1)); done; echo $i

real 0m16.043s
user 0m15.629s
sys 0m0.384s
1000001

# i=1; time while :;do [ $i -gt 1000 ] && break; i=`expr $i + 1`; done

real 0m1.411s
user 0m0.052s
sys 0m0.232s

Both let and (( )) perform calculations at almost the same speed. 'expr' being a process creation suffers heavily during calculations

Script to dynamically choose fd for a flock

This is a sample script which automatically chooses a unused fd, opens the requested lock file with that fd and acquires a lock on that file

#!/bin/bash

# function to acquire to flock on a local file
# Usage
# flock [-e|-x|-w ] 
# Return
# non-zero in case of errors
# flock_fd variable will contain the fd
function flock
{
 [ -z "$1" ] && return 

 local file="" timeout=20 mode="-x"
 while ! [ -z "$1" ]; do
  case "$1" in
  -x)
   mode="-x"
   shift
   ;;
  -s)
   mode="-s"
   shift
   ;;
  -w)
   timeout=$2
   shift
   shift
   ;;
  *)
   file=$1
   ;;
  esac
  ! [ -z "$file" ] && break
 done
 [ -z "$file" ] && return 1

 ! [ -f "$file" ] && touch $file
 ! [ -f "$file" ] && return 1

 local freefd=`ls /proc/$$/fd | sort -n | awk 'BEGIN{count=0} {if($1 != count) {print count; exit} else {count++}}'` 
 let freefd=$freefd+1

 eval "exec $freefd>>$file"
 [ $? -ne 0 ] && return 1
 flock $mode -w $timeout $freefd
 local ret=$?
 if [ $ret -eq 0 ]; then
  flock_fd=$freefd
 else
  eval "exec $freefd>&-"
 fi

 return $ret
}

# function to unlock a local flock
# Usage
# flock_unlock 
# Return
# non-zero in case of errors
function flock_unlock
{
 # close the file, it would remove the locks
 [ -z "$1" ] && return 1
 eval "exec $1>&-"
 return 0
}

SLES11SP1 pure-ftpd performance poor, uses mmap instead of sendfile

The default pure-ftpd that is available with SLES11SP1 seems to be using mmap for download instead of the preferred sendfile()

When you install the source rpm for pure-ftpd you would see that sendfile code is there with proper flags, and pure-ftpd does not seem to use it

If you check the pure-ftpd binary for any references to sendfile() in pure-ftpd binary, then there won't be any

$ nm /usr/sbin/pure-ftpd | grep sendfile

When you look deeper into the code there is a bug in src/ftpd.h of pure-ftpd bundled with SLES11SP1 which causes mmap to be used instead of the preferred sendfile()

Install the source rpm and check the sources and Create the source tree for the package. Unpack the source tar ball and apply patches

$ rpm -ivh pure-ftpd-1.0.21-183.11.2.src.rpm 

$ cd /usr/src/packages
$ rpmbuild -bp SPECS/pure-ftpd.spec

Once the sources the sources are installed, if we look at how sendfile can be enabled from Makefile/configure scripts

# grep SENDFILE * | grep LINUX | grep 64
config.h.in:#undef SENDFILE64_LINUX
configure:#define SENDFILE64_LINUX
configure.ac:  AC_DEFINE(SENDFILE64_LINUX,,[define if you have a linuxish sendfile64])

 But the ftp sources in src/ directory

# grep SENDFILE * | grep LINUX | grep 64
ftpd.h:# undef SENDFILE64_LINUX
ftpd.h:    defined(SENDFILE_HPUX) || defined(SENDFILE64_LINUX)
ftpd.h.orig:# undef SENDFILE_LINUX64
ftpd.h.orig:    defined(SENDFILE_HPUX) || defined(SENDFILE_LINUX64)

The configure/makefile scripts use SENDFILE64_LINUX, whereas the ftp sources use SENDFILE_LINUX64. Due to this incorrect macro variable name being used in sources, even though sendfile is enabled from configure/makefile level, the code generated with SLES11SP1 does not use sendfile().

http://bradthemad.org/tech/notes/patching_rpms.php contains details of how to make changes to the package and rebuild the rpm

A burn production: Never Extinguish

A burn production: Never Extinguish from Burn on Vimeo.

Tech to Improve Your Video Productions

TEDx in a Box

TEDx in a Box from IDEO.org on Vimeo.

Greenpeace - Ocean Action

Greenpeace - Ocean Action from Jens Blank on Vimeo.

New Look - Nap On The Bow

New Look - Nap On The Bow from Tim & Joe on Vimeo.

Sin City

Sin City from Luke Gibbs on Vimeo.

Mindshare "Storytelling"

Mindshare "Storytelling" from Andy Martin on Vimeo.

Space Oddity

Space Oddity from Andrew Ruttan on Vimeo.

We Cut Corners "Pirate's Life"

We Cut Corners "Pirate's Life" from Kijek / Adamski on Vimeo.

WAVE REFLECTION

WAVE REFLECTION from Koji Aramaki on Vimeo.

Plugs

Plugs from Father's Stash on Vimeo.

Marco Tempest: The magic of truth and lies (and iPods)

"Motion Design" Center : "What is Motion Design ?"

"Motion Design" Center : "What is Motion Design ?" from Motion Plus Design on Vimeo.

Wee Wise Words - Heaven and Hell

Wee Wise Words - Heaven and Hell from Flickerpix on Vimeo.

Innovation Starvation

"Most people who work in corporations or academia have witnessed something like the following: A number of engineers are sitting together in a room, bouncing ideas off each other. Out of the discussion emerges a new concept that seems promising. Then some laptop-wielding person in the corner, having performed a quick Google search, announces that this “new” idea is, in fact, an old one—or at least vaguely similar—and has already been tried. Either it failed, or it succeeded. If it failed, then no manager who wants to keep his or her job will approve spending money trying to revive it. If it succeeded, then it’s patented and entry to the market is presumed to be unattainable, since the first people who thought of it will have “first-mover advantage” and will have created “barriers to entry.” The number of seemingly promising ideas that have been crushed in this way must number in the millions."

http://johniac.posterous.com/innovation-starvation-world-policy-institute

Julian Treasure: The 4 ways sound affects us

Harald Haas: Wireless data from every light bulb

Adam Ostrow: After your final status update

MAP_POPULATE performance with ext3

Performance of mmap_read without MAP_POPULATE, around 90 MB/sec

XXX_01:~ # ~/a.out -i /tmp/test/testfile
Time taken : 22 sec
Data Read : 2097152000 bytes
Speed is : 90.909091 MB/sec
XXX_01:~ #
XXX_01:~ # umount /tmp/test; mount  /dev/vx/dsk/vxvmdg/fs_iscsi_ext3 /tmp/test
XXX_01:~ # ~/a.out -i /tmp/test/testfile
Time taken : 22 sec
Data Read : 2097152000 bytes
Speed is : 90.909091 MB/sec
XXX_01:~ #

Performance wih MAP_POPULATE, around 86 MB/sec

XXX_01:~ # ~/a.out -i /tmp/test/testfile
Time taken : 23 sec
Data Read : 2097152000 bytes
Speed is : 86.956522 MB/sec
XXX_01:~ #
XXX_01:~ # umount /tmp/test; mount  /dev/vx/dsk/vxvmdg/fs_iscsi_ext3 /tmp/test
XXX_01:~ # ~/a.out -i /tmp/test/testfile
Time taken : 23 sec
Data Read : 2097152000 bytes
Speed is : 86.956522 MB/sec
XXX_01:~ #

Looks like the built-in read-head of ext3 or linux is good enough without the need for MAP_POPULATE when used with mmap

James altucher on atheism

Altucher on atheism 

ATHEISM
@mczirjack asks: What are your thoughts (if any) of the expanding Aetheist movement i.e.: Sam Harris, Richard Dawkins, etc
ANSWER:
Atheism is almost a one-word oxymoron. It’s an organized religion against organized religions. But they still try to keep all of the trappings of an organized religion: every “professional atheist” tries to lay out an ethical system.
I could think of myself as an atheist also – I don’t believe in a man with a beard who magically created the Universe. Then I can lay out an existential system of ethics and ways for men to deal with each other without the words “under God” hanging over them.
Most people forget that Buddha was an atheist. And that even in orthodox Judaism there is no real word for God.
I prefer,for myself, to develop a system of happiness, to eliminate the constant brainwashing that occurs around me, and to try to enjoy life today.
In terms of the question: “do I believe in a higher power?” I would have to answer that I do believe in the concept of “surrender” which may or may not imply a higher power (who knows?). In other words, many situations get so difficult you want to throw up your hands and just say, “you know what, I did all I can. I leave the rest up to you.” And who is that you? It might be a higher power. It might be a creative force inside of you that is dying for those moments to be unleashed. Or it might simply be the feeling of gratitude that is always worth cultivating to help one find more happiness in life.

Meltdown - The men who crashed the world

Corruption in media

Desperate measures to finance escalating costs of production are also happening because hordes of players enter the media sector for a variety of reasons. There are no less than 40 news channels across the country financed by political parties or families, according to this documentary. A highly fragmented market that shows no signs of consolidating. 

The more expensive news gets to produce, and the less advertising there is to go around, the more shows you will get on gadgets and cars and movies. And fewer news crews going off to the countryside to report what is happening to ordinary people. Not reporting is not a cognizable offence, but it undermines the reason for the existence of journalism in a free society. 

And lots of other insightful comments at http://www.thehindu.com/arts/magazine/article2475946.ece

Comment on slashdot about speed of light by MichaelCrawford

I am intimately familiar with the interaction of light with matter as a result of having been an avid Amateur Telescope Maker [geometricvisions.com] and Amateur Astronomer since the tender age of twelve.
This led to my acceptance to study Astronomy at Caltech in the Fall of 1982, where I was privileged to attend a non-credit class called "Physics X" that was taught by The Immortal Richard Feynman. You could ask him any question you wanted - it didn't have to be about Physics even - but the ensuing discussion had to be purely conceptual. Questions that would require Feynmen to work out equations on the chalkboard were not permitted.
One afternoon I pointed out to him that the phenomenon that light slows down as it passes through a medium just had to be wrong. When one examines any medium at a subatomic scale, it is mostly empty vacuum with some rare particles that have all been either proven or are suspected to be geometric points. (While Protons and Neutrons have a non-zero diameter, they are each composed of three quarks, which themselves are thought to be point particles.)
"Surely," I pointed out to Feynman, "When light passes through all this vacuous space inside a piece of glass, it always travels at precisely C! How could Snell's Law" - which yields the angle of refraction when light passes through the surface of a medium - "possibly be correct!"
I knew damn well that Snell's Law was correct, as Snell himself experimentally demonstrated the law hundreds of years ago. While he did not measure what the Speed of Light had to do with refraction, we have been able to measure light's speed for over a century.
Feynman replied that when light passes through matter, the charged particles in that matter oscillate in sympathy with the oscillations of the light's electomagnetic field. But because they are all in a bound state, and because accellerating charged particles causes them to emit light of their own, thereby carrying away energy and so dampening their sympathetic oscillation, the movements of the charged particles in matter is not quite in phase with the waves in the light passing through the medium.
Feynman concluded, "The light emitted by the charge particles in matter interferes with the light passing through the medium" - that is, wave peaks add to wave peaks, and so with troughs, while peaks and troughs together cancel each other - "so that the resulting combination of light waves only appears to move slower than C."
Thus the Photons are always moving at a constant velocity of C, but all the Photons in the medium interact so that passing a Photon through the medium will result in the exit Photon being delayed from the timing you would expect from when the entrance Photon entered the front surface. They key to understanding all this is that the entrance and exit Photons are NOT THE SAME PHOTON!
Feynman discusses this in a really lucid way, with rigorous mathematics, in Volume II of The Feynman Lectures on Physics. Volume II covers Electricity and Magnetism, Volume I covers Classical Mechanics - Newton's Laws of Motion and such - while the third volume does Quantum Mechanics. The set of three is expensive but are easy to read, even if you don't know much Calculus, and would be a good investment for any Slashdotter.
I was mortally embarrased to realize years later that I had asked Feynman a really basic, purely conceptual question whose completely rigorous answer led to him sharing the 1965 Nobel Prize with Tomanaga of Japan! Their Quantum Electrodynamics describes the interaction of light with electric charge with complete precision.
Feynman's formulation uses a conceptual drawing called a Feynman Diagram as a calculational and explanatory device. I don't know how Tomanaga formulated his Quantum Electrodynamics, but my understanding as that at first no one could understand why the two theories seemed quite different but always yielded the same numerical results. Some time later Freeman Dyson - Esther Dyson's father - published a paper that demonstrated that their two theories were in fact equivalent. I expect that it was Dyson's paper that clinched their Nobel.
Everyone who knew anything about Dick Feynman - not just us Tech students, as he was at Cornell before Caltech - considered him a heaven-sent deity because throughout his life he considered it far more important to teach Physics than to understand it. The Feynman Lectures resulted from a year he spent teaching Freshman Physics. Some of his lectures were filmed; I expect you could buy DVDs, or maybe find them on YouTube.
Over the West entrance of the Dabney Student House at Caltech is an elaborate, fantastical sculpture of Heaven. God's face looks just like Feynman's!
Some of the happiest memories of my whole life are of the times I spent not just being taught by Dick Feynman, but getting to know him as a person. Such an opportunity doesn't come to many. For having had that opportunity I am truly priveliged.

The Alphabet 2 from n9ve on Vimeo.