Add a Simple Samba File Server as a Domain Member

ssambaIf you already have an Active Directory Domain Controller in place, diligently servicing all your needs and making itself indispensable, hopefully you’ve chosen Linux and Samba 4 to fulfill this.

If you haven’t used this free and open version of Active Directory and the domain controller, perhaps you’d like to? For the latest and greatest Samba 4 version that you compile yourself, you can follow the steps outlined for Debian Wheezy (and possibly Ubuntu).

Remember, too, that Debian backports now has a much more recent version of Samba 4 in its archives, and it seems to be working great right now.

But suppose you have that beast in place, and you want to add, say, a network file server — some storage that can be accessed by all domain users and whose rights and permissions are determined by your Active Directory domain controller.

It’s easy to just create a share on your Samba 4 Active Directory Domain Controller, and serve that out, with all the permission “goodness”.  But maybe you don’t have tons of storage in that box, or in that VM, and you don’t want to, either.

Or, you could find some filthy user’s computer in the domain that has a lot of hard drive space in an array, and map it out to the others. (I mean lovely user – it’s a joke!#$@!). But we know that’s a terribly unsettling idea.

So why not build out or spin up a new Samba 4 server that’s pretty much dedicated to housing user data, whether that’s shared between the masses for apps or data, or just for daily backups.

I’ll tell you why. Because you’re frightened. How can you get one Samba server to listen to another Samba server and believe all its tales about the users and their permissions? How will the UIDs and GIDs match up for filesystem stuff? Sure, you know it’s possible. You’ve looked around. You’ve seen all kinds of insanity for linking Samba 3 into Windows AD/DC, mapping local users and huge cut-‘n-paste swaths of wild configuration blocks.

But if you go Samba 4 again, will you find any documentation that is currently to the point where Samba 4 development is? Or to the way that Debian has mutated it?

The answer is yes! All over the place, including the Samba site itself, and it’s all whacko and incomplete, if all you want to do is create a simple file server – you just want a simple Samba 4 file server that is a domain member file server. You don’t care if users can ever log into the Linux box. You just want them to have access to Windows file shares served with Windows users permissions and rights honored.

And you may think… I’ll just make a big RAID disk on some server, and serve out to my One Box To Rule Them All Samba 4 AD/DC – and then it has big drives and plenty of space! Well, it would if NFS or CIFS handled extended filesystem attributes… Alas! We are thwarted.

So here’s what I did:


You must a Samba 4 Active Directory Domain Controller running just fine already. Or a normal Windows-y one if you must run a Windows one for some nonsense reason.

You must have a Debian (or possibly Ubuntu) server ready to go with only the minimal stuff installed – like the SSH server. That’s because it’s proper. And you will be proper. Another distribution is fine of course, just don’t complain to me.

Don’t use Debian Wheezy’s version of Samba 4. It’s not ripe yet. And don’t use Samba 3. It’s overripe. (Simply too many notes) Either roll your own Samba 4 from source or enable the Debian backports repository and go with that version of Samba 4, there.

If you follow the instructions to roll your own, skip the samba-tool domain-provision step and the Cold, Cruel Kerberos sections!It makes your new Samba 4 server have delusions of grandeur and it won’t want to listen to your already-existing AD/DC.

If you go with Debian backports, this is what you’ll need:

# apt-get -t wheezy-backports install samba samba-doc samba-testsuite winbind libnss-winbind
# apt-get install acl

Setting Up the Samba 4 Domain Member

The smb.conf [global] section

If you’re using the Debian version of the /etc/samba/smb.conf from backports, throw away everything in it, because it’s garbage Samba 3 stuff and they haven’t bothered tidying anything up.

For a simple Samba file server, you just need your [global] section and your share definition. I’ll highlight some of the stuff in it, after this example listing (that works just great). Be certain that your filesystem that serves out the stuff is mounted to support xattr and acl’s. (in your /etc/fstab put the mount options “user_xattr,acl” in place of “defaults”).

Anyway, here’s that /etc/samba/smb.conf file:

  netbios name = <servername>
  workgroup = <win domain>
  security = ADS
  realm = <kerberos realm>
  encrypt passwords = yes

  idmap config *:backend = tdb
  idmap config *:range = 70001-80000
  idmap config <win domain>:backend = ad
  idmap config <win domain>:schema_mode = rfc2307
  idmap config <win domain>:range = 3000000-4000000

  winbind nss info = rfc2307
  winbind trusted domains only = no
  winbind use default domain = yes
  winbind enum users = yes
  winbind enum groups = yes

  vfs objects = acl_xattr
  map acl inherit = Yes
  store dos attributes = Yes

That’s it, really. I love Samba 4 for this. Consider me gushing and oozing again, all over the Samba development team again for doing such a damn fine job pulling so much together. Very well done.

Here’s what some of those things are:

  • netbios name – anything you like – I like to use the hostname portion from DNS
  • workgroup – that’s your Windows “domain”. Make it the same as your main Samba 4 server’s.
  • realm – your Kerberos realm – again, make it the same as what your main Samba 4 server’s is.

Now, the idmap stuff should be fine for you to use, if your AD/DC is standard Samba 4. The range values are UID values that map to local users (the asterisks) and your AD users (the <win domain> ones).

By default, at least right now, Samba 4 is by default using the UID range of 3000000 – 4000000. That can be changed when you set up your Samba 4 AD/DC. A Windows-based AD/DC is probably different. Find it out, and put it there.

The local mapping range just shouldn’t overlap your domain range ones. And I kept mine well above any UIDs I’d really be using locally on this machine (in the 70000’s… 😉 )

I hate winbind. I just copy and pasted that bit from some Samba site documentation dealing with domain member servers. It seems necessary, especially the nss bit for letting your system know about domain users on the AD/DC.

The last 3 lines are critical if you want permissions to be right on the filesystem here.

The smb.conf Share Definition

Behold the glory of letting a fascist AD/DC handle all your decisions for you!

   path = /srv/storage
   read only = no
   admin users = "@MYDOMAIN\Domain Admins"

Whatever path you want there, Sparky. And the MYDOMAIN bit is your domain of course, just like above. And oddly, this is the tricky one that caused me quite the headache.

After I got all this set up, I couldn’t set permissions on the share as a domain administrator. Permission denied. You have to add the “admin users” parameter even for domain administrators. I like this because it gives your local root the final authority still. But nobody mentioned this and it took forever to figure out. Here, I have the admin users be anyone @MYDOMAIN who is part of the Domain Admins group on the AD/DC.

But I digress.

The Final Steps

Of course, reload or restart this Samba daemon and its associated daemon cohorts of nmb and winbind.

# service samba restart
# service winbind restart

I hope you didn’t “provision” this Samba 4 server with samba-tool…

Also, remember to edit your /etc/resolv.conf file to make sure the nameserver you’re using is your Active Directory Domain Controller. If you don’t, you won’t be able to join the domain, since it relies on that absurd DNS injection flotsam.

The next step is to join your domain as a domain member.

# net ads join -Uadministrator

No, it’s not magic. It just gets your domain info from you smb.conf file.

If you got an error, you might want to make sure that you’re using your main AD/DC server as your DNS server in /etc/resolv.conf – that is, make sure your ‘nameserver’ is set to the IP address of your primary AD/DC. If it’s not, you won’t get the revolting Microsoft DNS injection they made necessary, and you’ll fail.

Remember the NIS/YP stuff? No? Well, then just do what I say:

# edit /etc/nsswitch.conf

Then add “winbind” to the end of the passwd and group lines, while leaving the rest of the file alone:

passwd:         compat winbind
group:          compat winbind

Final Thoughts

You might want to reboot. Make sure your mountpoint for the share you’re offering has those attributes specified in /etc/fstab.

You’ll probably want to grant your domain admins the ability to control files and permissions on your shares as well:

# net rpc rights grant 'MYDOMAIN\Domain Admins' SeDiskOperatorPrivilege -Uadministrator

But maybe not! I think it’s pretty well handled by specify the admin on the share level of the smb.conf file. But I did both, because I was flailing with that problem I mentioned earlier, and will not reiterate out shame.

Anyway, that’s about it. Except for, of course, always be sure you install ntp since time is so critical for this implementation with Kerberos. Just a few steps really, but lots of words and background info on why. Hope it helps.

Change Default SMTP Relay Port in Debian’s Exim4

It seems fairly common for someone to have a private range of IP addresses behind a dynamic IP address assigned by an ISP. If this is your situation, you may get your SMTP port blocked by your ISP.

For those of us with SMTP relays in a central place on the Internet, having our SMTP port blocked by our “conscientious” local ISP is troublesome. But, they usually excel at troublesome.

So when your local machines or servers need to send mail, like, say, to report that a hard drive in the array has failed… they’re out of luck unless you send it through your ISP’s relay.

Unless… your ISP allows at least some mail ports though (or you set yours to listen on bizarre ports, which is commendable when necessary).

So we know that residential Comcast blocks SMTP port 25 which keeps us from relaying our valid email from local machines. But they don’t block port 587, which they consider “secured” for some reason. Why? I don’t know. You can, and should, and most smart people do encrypt on port 25. And you don’t have to encrypt on port 587 if you don’t want to. And relays can be open on port 587 as easily as port 25. So… not sure why they think port 587 is “secured” while port 25 “unsecured”.  I think they just enjoy being fascists all-around. (please don’t smite me Comcast, I’m just a poor thing trying at humor)

Anyway, mail servers typically aren’t configured to relay on ports other than 25. It’s pretty easy to get them to listen and relay on the other ports, though. This post isn’t about listening, though. It’s about sending. And to send mail, relaying on port 587 (submission port) instead of port 25:

# edit /etc/exim4/update-exim4.conf.conf

Then just change your SMTP smarthost (mail server that relays mail on  your behalf to its destination) line:


You just append 2 colons and the port number. Of course, your mail server actually has to be listening on that port as well. Debian’s (and by extension Ubuntu’s) mail server Exim4 automatically deals with protocol and encryption negotiation.

Remember, any time you change your update-exim4.conf.conf file you need to run:

# update-exim4.conf
# service exim4 reload

That lets Debian generate all it’s Exim4 configuration magic that vexxes so the Exim4 developers. But believe me, it’s nicer than having to worry about doing it all by hand in the pure Exim4 way.

By the way, you can also just reconfigure Exim4 using the standard Debian dpkg scripts, and for your “smarthost” question, answer with those extra 2 colons and the port number as well as the FQDN of your mail relay.

# dpkg-reconfigure exim4-config

That script stuff will also restart the exim daemon for you.

Do that, and your boxes can now happily relay to your central SMTP mail server on port 587 instead of port 25 – or whatever other port your preferences or necessities might take you.

Override DHCP assigned DNS Server and Domain Search in dhclient

Sometimes, particularly if you are on a residential broadband Internet connection, your Linux box might need to get its external IP addressed assigned by your ISP.

In this case, and in the case of Debian, you define your network interface with the dhcp flag in /etc/network/interfaces

auto eth0
iface eth0 inet dhcp

When you do this, by default, your /etc/resolv.conf file will be overwritten by whatever your ISP wants to assign you for your DNS servers and your search domain as well. It’s not always desirable that your /etc/resolv.conf file gets overwritten.

In Debian, if you are not using Network-Manager, that is, if you have a nice, minimal system, like for a router, it is the “dhclient” program that is handling the task of getting your IP address and network configuration information from your ISP.

You can, if you like, alter, override or ignore whatever your ISP is assigning you by editing the dhclient configuration file in Debian:

edit /etc/dhcp/dhclient.conf

In here, at the bottom, if you want to ignore your ISP’s name server assignment and use your own machine as the DNS server, you can “supersede” what the ISP’s name server gives you – that is, completely ignore the stupid thing:

supersede domain-name-servers;

That gives you just your local machine as your DNS server. So please do make sure you have one. But perhaps you still want to use your ISP’s name servers, but want to use yours as well, say, for example, reverse DNS entries in the ARIN black holes that you might locally use for your subnet. In that case, you can just “prepend” or “append” your own entry to what the ISP will assign:

prepend domain-name-servers;

That way, you get yours first, then whatever they want to give you.

Of course, you’ll probably want to assign your own search domain too, so you don’t have to go typing FQDN’s all the time and give yourself carpel tunnel’s syndrome. So here’s some preventative care:

supersede domain-search "";
supersede domain-name "";

If you do these things, then your /etc/resolv.conf file will be just how you like it, even with that presumptuous dhclient trying to make your machines believe everything it hears from your ISP.

Of course, you’ll need to bring the interfaces down and up to see the changes happen – just use the

# ifdown eth0
# ifup eth0

I would think that is self-evident, but I’ve been nagged at before for not saying such things. And I’m delicate.

Hope this helps!

Fix mdadm SparesMissing event detected

Sometimes when you create a RAID array using mdadm and do not specifically specify there are no spare drives available for the array, the mdadm monitoring daemon will report to you in daily emails that:

A SparesMissing event had been detected on md device /dev/md*

If you check your /etc/mdadm/mdadm.conf file, you’ll most likely find the ARRAY assembly line specifies an “spares” parameter that is non-zero.

To stop this daily nagging email message, just change the “spares” parameter to 0.

ARRAY /dev/md0 metadata=1.2 spares=0 name=<mymachine>:0 UUID=<myUUID>

The reload the monitoring daemon. In Debian:

service mdadm reload

Currently, that command will actually stop and start the daemon, rather than reloading it, but hey, who knows what the future modern world will bring?

Also, this is all assuming that you really do have no spares in your array, and the mdadm system just couldn’t believe such audacity, and concluded that you must have a least one that you just didn’t tell it about.

LVM Basics – A Quick Intro and Overview by Example

Storage technologies and methods can be a confusing subject when you’re first starting out, and depending on which route you choose, and how you organize them, these storage methods can continue to be confusing even when you know lots – unless you plan and organize well, at the beginning.

Take the time to think about what you want to have before losing yourself in the details of any given technology. And keep this in mind as you create.

Following is very generalized overview of some practical LVM use, presented in simple, no-frills terms. LVM tends to scare people away, seeming complex at first. And it can be, but it doesn’t have to be. This is the easy way, to get you started with LVM if you’re interested. You can make it harder (and perhaps better) later.

What is LVM? Why use it?

GNU/Linux gives you many options. LVM, the Logical Volume Manager is one of them. LVM lets you combine disks and partitions and even arrays of disks together into single filesystems.

LVM is very flexible. None of your storage media or arrays need to match up – you can combine a high performance PCI-e RAID10 array with an external USB 3.0 4TB hard drive if you like. Not always the best idea performance-wise, but often performance isn’t the most important thing.

LVM performs very well, too. Is your RAID0 SATA array running out of space? Add 2 more drives in another RAID0 and combine their capacities together. Or just throw in one new drive. It’s a messy way to do it, but you can. And it would still be fast.

Or perhaps you’re more sober, and want to create a SAN for your network, starting off with a 4-disk RAID10 with smaller drives. But you’re worried the space might fill up, and you can’t have the files span multiple filesystems. If you use LVM, you can just put in a second array at a later date, and expand your logical volume seamlessly.

 LVM Details

From a designer and administrator standpoint, you can think of LVM as having 3 main components:

  1. Physical volumes (devices or arrays).
  2. Physical volumes you bundle into made-up groups.
  3. Logical disks created and carved out of those groups you made.

This is where your planning comes into play (or not if you’re bad).

I’ve gotten into the habit of always using LVM to create the volumes which I will then format into filesystems. This gives me the flexibility to add more capacity, or take some away, at any point in the future.

The Debian distribution has long supported creating and installing itself into LVM volumes. Most of the major distributions now seem to support this as well.

So, using LVM, it’s a simple matter to take your 6TB RAID array and make a nice little root drive of 10GB for Debian, and a 5TB home directory, that you could then share with another 10GB partition you might make later for, say, Fedora.

Of course, you could partition your RAID array with extended partitions to achieve much of the same result, but you would not have nearly the flexibility later.

The Nitty Gritty

Working with LVM requires that you have the LVM tools. In Debian you can get these installed for you automatically if you choose to install onto an LVM volume, or you can get them later with

apt-get install lvm2

Instead of “normal” devices for hard drive volumes,  you’ll get logical devices that are create by the device mapper. In Debian, at least, they live under /dev/mapper and you also get a prettier version of the device names you create under /dev/<volume_group> — and you get to choose the <volume_group>

Creating Physical Volumes

The first thing you need to do is decide which devices you’d like to take over as LVM-controlled beasties. You can also specify individual partitions of disks if you like. The point being, you don’t have to even partition a disk first – you can specify the whole device if you like. Or partition it if you prefer. People will always argue over what’s best, and there is no clear winner. Which means, do what you like!

Let’s say you have 2 SATA3 drives, /dev/sdb and /dev/sdc. And you want to use them for LVM. The first thing to do is claim those physical volumes for LVM:

pvcreate /dev/sdb /dev/sdc

And lets suppose later you decide you also want to use your RAID 1 drive (/dev/md0) for LVM as well. You can designate any physical volume to be used with LVM, at any time.

pvcreate /dev/md0

Oh, I also have a USB 3.0 drive I might want to do something with:

pvcreate /dev/usb_drive_0

Do be careful though – consider any data on these destroyed.

Creating Volume Groups

Once you have some physical volumes in your system designated for LVM use, you can start grouping them up together. Creating volume groups doesn’t give you anything you can format into a filesystem, it just groups together whichever physical volumes you want to use into a named group that you can reference them by as one.

Think of it as the first abstraction layer — which of these physical devices do I want to group together and use as a big pool of space, from which I can make my littler hard drives.

You might want to consider how fast the drives perform when grouping. For example, you probably don’t want to group your super fast RAID array in with your USB drive. But maybe you’re fine grouping your RAID array in with your other SATA drives.

Yes, yes indeed, we’re fine with grouping our RAID in with our SATA drives. It will be fast. And the USB will be slow. How about we name these groups to reflect the fast and slowness of each. Maybe I can use the slow for backups?

vgcreate fast /dev/md0 /dev/sdb /dev/sc

That’s it. Easy to create a volume group. Just pick a name, like “fast” as we did here, and then list the volumes you want to use in it — ones that you created with pvcreate. And if you’re wondering, yes you can skip the pvcreate part above — but don’t — until you’re very sure that all this structure stuff is going to stick in your head.

Now, even though we only have one device in our USB drives, we may later add more nonetheless. So why not put it into an LVM group now – then later if we need to expand our backup area with more space, we can add a second USB drive, and just add it to this same group for instant gratification.

We’ll call this one slow:

vgcreate slow /dev/usb_drive_0

So we now have two spaces we can work with – the “fast” space, and the “slow” space. These are the volume groups. And we no longer have to care about which devices make them up. Well, until something goes wrong.

I suppose it’s worth noting that LVM provides no redundancy or backups, by default, although there are some great mechanisms built into it for doing so later, in some… interesting ways. So it’s really best to make sure your physical volumes have the level of redundancy and protection you want, before making those physical volumes into logical ones. I almost never make an LVM volume out of anything but a RAID array, unless I just don’t care that much about losing data, because I’m so awesome about backups (mm hmm).

Creating Logical Volumes

Now you can create logical volumes in your volume groups, and these logical volumes you can format just like hard drive partitions.

When you create your groups, LVM will allocate all the space possible on those devices, and make it available to you to create your logical volumes in. If you ever want to see how much space you have:

vgdisplay fast

You’ll get to see how much space you have, and also how much space has already been allocated to logical volumes.

Now, lets say I want to have a 100GB “disk” for my database files. Fast disk access is important for database work, so we’ll put that on the “fast” volume group:

lvcreate -n database -L100G fast

This will create a logical device of 100GB size called /dev/fast/database (at least in Debian). A more pedantic device name is also created called /dev/mapper/fast-database but I prefer using the first naming convention.

You can then use this newly-created device just like you would any hard drive. You can partition it if you like, or you can just create a filesystem on it:

mkfs.ext4 /dev/fast/database

Now, if you decide you just must partition these, they can be a little tricky to mount. Using a tool called partx you can extract that partition table out into devices you can work with, but we won’t go into that just now, here.

But if you skip the partitioning bit, these logical volumes mount just like normal filesystems, except really, they can be coming off of any number of drives in your volume group.

mount -t ext4 /dev/fast/database /var/mydatabase

Easy as pie! (when you don’t have to make the crust)

The same holds true for your slow USB drive:

lvcreate -n backups -L 2T slow
mkfs.ext4 /dev/slow/backups
mount -t ext4 /dev/slow/backups /mnt/backups

Here we created a “drive” of 2TB called “backups”, formatted it, and mounted it at /mnt/backups. This would be our USB drive.

Adding More Drive Space to an LVM Logical Volume

Now that we’re demystified, and have been using our system just fine for weeks, we ended up filling up our /dev/slow/backups with backups. Let’s say our USB drive was 3TB. We only allocated 2TB to our logical volume for backups, so really we have 1TB left free in the “slow” volume group. We could see this with

vgdisplay slow

So, if we wanted, we could just increase the size of our logical volume /dev/slow/backups to eat up that extra 1TB that exists in that volume group, and so give us the full capacity of that USB drive:

lvextend -L+1T /dev/slow/backups

Here we give a relative size in the -L parameter, saying we want to add 1TB to the already-allocated size of 2TB, making a total of 3TB. Of course, this doesn’t make the filesystem bigger, just the “disk” the filesystem is on. But ext4 is easy to resize:

resize2fs /dev/slow/backups

By not specifying a specific size, resize2fs will just make the filesystem as large as it can on that “drive”. And you don’t have to take the disk offline either — this is an online resize, and I’ve never had it fail. But of course you should do it offline. But I never will.

You can also shrink filesystems and volumes, but I’m not going to go into any of that here because shrinking filesystems is unnatural.

But suppose even 3TB is not enough for your backups. Well, it’s easy to add a new device to a volume group, and make it available for logical volume to eat up in its gluttony.

You buy another USB drive! So now we need to add it to the “slow” volume group. You got a good deal on 4TB drive this time. So you have your original 3TB one, and now a 4TB one you’re going to use to expand your storage. Assuming it’s assigned /dev/usb_drive_1

First, create the physical volume, as usual:

pvcreate /dev/usb_drive_1

Then add it to your “slow” volume group:

vgextend slow /dev/usb_drive_1

After you do that, if you look with

vgdisplay slow

You’ll see that you now have a whopping 7TB volume group called “slow”.

Then you just do the same as above to extend your logical volume, or “drive”, for your backups:

lvextend -L+3T /dev/slow/backups
resize2fs /dev/slow/backups

Et voila! You now have a 6TB backup “drive”, because you didn’t use the full 4TB off the new one. You left 1TB free in the “slow” volume group, because you’re not always a complete hog, and you may like to use that 1TB for some other volume in the future, like:

lvcreate -n pron -L1T slow
mkfs.ext4 /dev/slow/pron
mount -t ext4 /mnt/relax /dev/slow/pron

Because, well, if not a hog, then perhaps a pig. Oink.  And that uses the last of the drives.


You can, of course, remove devices from LVM groups as well. You just need to make sure you have enough space in the volume group to do so. For example, if you have 2 USB drives, one 3TB and one 4TB, making a total of 7TB — if you’re using up 6TB in logical volumes, you won’t be able to take away either of those USB drives, unless you can shrink down your logical volumes first (after shrinking your filesystem first, unless you like agony).

The point being, LVM is quite flexible.

One of my other favorite features is the ability to take a live snapshot. This is great for backing up whole disk images of a live-running system. You create an LVM “snapshot”, and then you can dump that image anywhere you like, and the filesystem will be in a consistent state, even with the system running. It’s wonderful for virtual machine images especially.

But I’m not going into that either here, just  yet.

Hopefully this might have helped you a little. I remember when I was first looking at LVM it was a confusing mish-mash of all these different options, and nothing spelled it out simply. Hopefully, this has managed to do so, if you’re looking to get started with it, for play or production.

And as always, check out the man pages. There are lots of other places out there too with more detailed and specific use cases and feature examples.

Honestly, using LVM has changed everything for me. It’s certainly worth looking at. Best to you!

Fix for MRTG Generating SNMP_Session Error in Debian Wheezy (and possibly Ubuntu)

Lately, after an upgrade from Debian Squeeze to Debian Wheezy, MRTG is sending emails every few minutes when it runs from the crontab. This is quickly filling up my Inbox. There error message is as follows:

Subroutine SNMP_Session::pack_sockaddr_in6 redefined at /usr/share/perl/5.14/ line 67.
 at /usr/share/perl5/ line 149

After waiting for a while to see if a fix came from Debian, I decided to look around on my own. It seems there is a patch that works, but it has not been propagated out to the repositories. Luckily, this error problem is easy to fix.

You can apply the patch at the link above, or you can just edit a file and make 2 quick changes: Edit the file


Change line #149:

old: import Socket6;
new: Socket6->import(qw(inet_pton getaddrinfo));

Then change line #609:

old: import Socket6;
new: Socket6->import(qw(inet_pton getaddrinfo));

That seems to fix the problem quite well. Hopefully the Debian maintainer will get that change in sooner that later so others don’t have to bother!

Note: Someone commented that the line numbers listed were a bit off from version to version of Debian. Not entirely unexpected. It’s the change to calling the class directly that counts.

How To Deal With Udev Children Overwhelming Linux Servers With Large Memory

A few months ago a client purchased a new server and asked me to set them up with Linux-based virtual machines to consolidate their disparate hardware. It was a big server, with 128 Gigs of memory. I chose to use Debian Wheezy and QEMU KVM for the virtualization. We’d be running a couple Windows Server instances, and a few Debian GNU/Linux instances on it.

Unfortunately, we ended up encountering some strange problems during heavy disk IO. The disk subsystem is a very fast SAS array, also on board the server, which is a Dell system. The VM’s disks are created as logical volumes on the array using LVM.

Each night, in addition to the normal backups they do within the Windows servers, they also wanted a snapshot of the disk volumes sent off to a backup server. No problem with LVM, even with the VM’s running, when you use the LVM snapshot feature. This does tend to create a lot of disk IO, though.

What ended up happening was that occasionally, every week or two, the larger of the Windows server instances would grind slowly to a halt, and eventually lock up. The system logs on the real server would begin filling with timeouts from Udev – about it not being able to kill children. This would, in turn, effect the whole system – making a reboot of the whole server necessary. Very, very ugly, and very, very embarrassing.

I tried a couple off-the-cuff fixes that were shots in the dark, hoping for an easy fix. But the problem didn’t go away. So I had to dig in and research the problem.

It turns out that Udisk (which is part of Udev) decides how many children it will allow based upon the amount of memory the server is running. In our case, 128G – which is quite a lot. This number of allowed children was a simple one-to-one ratio, based upon memory. However, with this much memory, that many children seemed to be overloading the threaded IO capacity of this monster server, causing blocks, during live LVM snapshots being copied.

What I ended up doing was manually specifying that the maximum number of allowed children for Udev would be 32 instead of the obscene number the inadequate calculation in the Udev code came up with. Since doing this, the server has run perfectly, without a hitch, for a good, long time.

So this is for anyone who may have run into a similar problem. I could find no information about this on the Internet at the time. But I did manage to find how to effect the number of children Udev allows. You can do it while the system is running (which will un-happen once the server is rebooted) or you can put in a kernel boot parameter to effect it, until the Udev developers fix their code to provide a sane value the maximum number of children allowed in systems with a large amount of memory.

At the command line, this is how. I used 32. You might like something different, of course.

udevadm control --children-max=32

And, as a permanent thang, the Linux kernel boot parameter is “udev.children-max=”.

Hopefully this will save some of you some of my headache.

Compiling Samba 4 on Debian Wheezy – Active Directory Domain Controllers Ho!

ssambaI’ve managed to avoid working with Microsoft’s Active Directory for many years, which is actually somewhat of a skill. But recently a client, unhappy with the support and the direction their MS “specialist” was taking them, asked me to see what I could do with their network.

Long ago I advised them to steer clear of Active Directory if they could, because it would only tie them in to more and more expensive MS “necessities” over time. This is the position they found themselves in, years later, having to shell out more and more money to MS and their MS-oriented “consultant” just to keep things running – and not running well, either.

It was important to this company that they remain able to manage user identity and authentication from a central place, as well as authorities and permissions. So I thought it might be a good time to at last examine Samba-4 and its claims to support Active Directory.

The Samba-4 guys can claim anything they like related to Active Directory and I would be none the wiser. I knew nothing of AD. But that soon changed as I delved into Samba-4. I must point out that the things I say here are my own impressions and conclusions based upon next to no research – so I could be quite wrong in some places.

It turns out that Active Directory is an unholy marriage of DNS, Kerberos, LDAP and CIFS. Unholy only in that it tries to obscure the individual technologies. On the MS side of things, they like to include DHCP, but it isn’t necessary at all.

Maybe I shouldn’t say that it tries to obscure the individual technologies. Maybe I should say it tries to unite them in holy simplicity for the good user. Yes, that’s it.

The tricky key (and shackle) is DNS. I always wondered why Windows clients had to use the Active Directory server as their DNS server – it seemed so limiting (and error-prone). It turns out that Active Directory will “inject” funny yet specific DNS names into your domain that identify the AD server to clients. It’s not necessary to be designed that way of course, really – but it’s a good hook. Windows clients joining a “domain” expect these funny DNS entries, and it does no good just specifying the AD server to connect to, unless you have these DNS entries being injected there as well. (salutes and rifle fanfare, etc.)

As for Kerberos and LDAP – anyone who’s worked with them knows it can take some strenuous wrestling to get stuff seated and right for handling your user auth stuff. And in this I am actually impressed with Active Directory. MS has done a great job integrating these Free technologies into something standardized on a platform. Although there are many ways this can be accomplished, Microsoft’s dominance on client machines made a standardization possible. And I’m happy that the European courts saw fit to rule in a way that allowed these Free technologies to be free once again — and this is where Samba-4 comes in.

If you have worked with Samba in the past, you know how versatile it is for file serving, and how complicated it can get. I don’t think I’ve ever dealt with a longer man page with more options. Samba 4 is no different. However, in some ways, it’s much easier than Samba 3 if you’re using the standard Windows administration tools to administer the users and shares. From my understanding so far, you basically just put the shares you want into the smb.conf file with minimal definitions, and define the user authority stuff through the Windows tools connected as an Administrator to Samba 4. If you’re managing rights on share servers other than your Samba 4 DC, then you don’t even have to worry about defining them in the smb.conf file.

But of course you can if you want – there is a command line tool that gives you access to the same stuff that tweaks this marriage of Kerberos, LDAP and DNS – without the need of Windows at all.

Anyway, enough of these background thoughts. The Samba team has done a great job. A really great job. And I’m going to donate some dollars to them, because they do need pizza, even though they say they don’t.

So, being mostly a Debian guy, I decided to try this Samba-4 out in Debian Wheezy. The Wheezy repositories have an older version of Samba-4, of course. This is one of those rare instances where I will compile my own version of a package outside the normal Debian space, since Samba-4 is such a newer and only recently became stable, in the more unix-y sense of stability.

And it’s not that hard to compile and get Samba-4 running in Debian Wheezy. And it’s certainly worth the time if you want to replace an Active Directory Domain Controller with Samba-4 or to just play with it, to see what it’s all about. I took some notes while I was doing it, which I decided to share here, since other people have found my doing so helpful previously, on other systems.

Note: It looks like Debian Backports is updated with a newer version of Samba4 at last. This is a great way to go to avoid compiling and maintaining your own. I’ve tried it, and it works well. FYI

Do Your Debian

I used a KVM virtual machine to create a Debian Wheezy installation that would run Samba-4. I think it’s probably a good idea not to use a production server at first. If you use a VM, you can always just trivially put it into production later.

During the install, I chose the most minimal installation package option with the addition of an SSH server.

Of course, this will probably work just as well with other distributions if you get your library dependencies right. Ubuntu may work with no modification, but I’m not sure.

Kerberos is very finicky about time. You will need an ntp server to keep your clock well synchronized.

apt-get install ntp

Also, generally I like to assign my servers static IP’s. And it also seems like the AD stuff does not like changing IP addresses once it’s been set up. Seriously. It’s probably an ingredient in the unholy glue.

edit /etc/network/interfaces

Change your “dhcp” flag to “static” and give yourself your proper address and routing info.

auto eth0
iface eth0 inet static

Unless you’re right on top of your DNS zone information, including PTR records, you should probably edit your /etc/hosts file too, to include the machine name you’re going to use:

edit /etc/hosts

I’m not really sure about the entry here, but it freakishly seemed to work for me. And I’m not sure why I did it. And it may not be necessary. I think it must not be.       localhost    samba    samba

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

As for DNS, you can use Bind9 just fine with Samba 4 — but Samba 4 also has its own built-in DNS server that does that filthy injection. If you want to use Bind9 as your backend DNS server, you can, but you will need to allow the Samba 4 server to dynamically update the zone for your domain with Kerberos. There are howto’s on that. I chose to just let Samba 4 use its own built-in DNS server. Because I’m lazy. And I’m just playing for now. And I don’t like a “domain controller” being able to update my real DNS zone file.

This leads to an interesting, and by that I mean boring and unnecessary, discussion of how you should name your Active Directory “domain”. There are a few schools of thought on it, and even Microsoft has changed their tune over time on the subject. I have chosen to name my Samba 4 “domain” as a “subdomain” of my root domain – that way the Active Directory stuff doesn’t have to be authoritative for my whole domain, and I don’t have to make up a fake domain either.

And leave it to Microsoft to terribly confuse everyone by “making it easy”. By domain they do not mean a DNS domain. It’s a hybrid abomination of DNS and what is known in Kerberos as a “realm”.

So yes, well, I made Samba 4 be the DNS server, but it will also do sensible lookups to the real DNS information from my proper DNS server when it doesn’t know a name. That’s why I named it as a DNS “subdomain” (host) rather than the whole domain. For resolution:

edit /etc/resolv.conf

Now, in Ubuntu you’re going to have to do some special editing of configs to keep Network-Manager from overwriting your resolv.conf file after you make these changes.


The first should be your Samba 4 installation IP. The second should be your real DNS server.

Probably quick & dirtiest to reboot after all this, if you like that sort of thing. BTW – make sure your /etc/hostname matches your DNS hostname. I don’t know if it’s necessary, but how can you stand it otherwise??

Debian Requirements to Compile Samba 4

I should mention, if you plan on having your Samba 4 server also be a filesharing server, and for the Active Directory stuff to manage the users and permissions for you, you need to make sure that whatever filesystem you’re going to be serving out is supporting ACL’s and extended attributes. In Debian this is a normal part of their ext4 mounts, and I think their ext3 ones as well. So you’re set!

But still, might be good to put it in, in your /etc/fstab, just as a reminder. Do, of course, use your own partition’s UUID. And whatever mountpoint you want to share.

UUID=b99750a8-9c39-11e3-82f1-525400990c6c   /home ext4      user_xattr,acl  0       2

Many docs also want you to specify barrier=1 as a mount option, to make sure stuff doesn’t get corrupt in a power failure. This is enabled by default in ext4, but you may want to in ext3. And if you’re using LVM volumes, this is passed through and respected now. Ah, the wonders of the modern world.

Now, what you really want to know: which Debian packages do I need to install when compiling Samba 4? Well, how about these?

apt-get install build-essential pkg-config libacl1 acl libacl1-dev libblkid-dev libblkid1 attr libattr1 libattr1-dev libgnutls-dev libreadline-dev python-dev python-dnspython gdb libpopt-dev libldap2-dev dnsutils libbsd-dev krb5-user docbook-xsl libcups2-dev libncurses-dev libpam0g-dev libdm0-dev libfam0 fam libfam-dev xsltproc libnss3-dev docbook-xsl-doc-html docbook-xsl-ns

If you don’t have other Kerberos servers, well, I just used this server as my kerberos ones, and it works just fine. The initial realm, where it defaults to your domain name in upper-case — I made that the FQDN in upper-case as well. Apparently the realm likes to be upper-case.

Maybe you’ll want to reboot again, after the acl stuff. Maybe not. Maybe you didn’t reboot a few minutes ago, so it will only be this one reboot. Or none. I don’t care.

Compile Samba 4

The version of Samba I grabbed was their latest at the time, listed below. They may have a newer version when you read this, so always check the Samba site for the version you want.

I like compiling in /usr/src — and I’m letting Samba 4 install to its default location, which I know is a horrific violation of Debian policy. But I’m naughty.

cd /usr/src
tar -xzf samba-4.1.4.tar.gz
cd samba-4.1.4
./configure && make && make install

Oh, the places we’ll go.

After that completes successfully the first try and love descends upon all humanity, you might want to put the install directory into your PATH environment variable so you can avoid over-stressing your poor little phalanges. Put this in your .bashrc

export PATH=/usr/local/samba/bin:/usr/local/samba/sbin:$PATH

If you’re feeling particularly cavalier, trusting in the goodness of strangers that is. And source it! (or log out/in, open a new terminal, whatever)

I also symlinked my /usr/local/samba/etc to /etc/samba to make it less typing to edit configs:

ln -s /usr/local/samba/etc /etc/samba

Then you’ll want to make the Samba 4 stuff work. Right? First thing is to provision the so-called domain. I’m leaving it open to do some Un*x-side integration later here – that’s why the “rfc” switch.

samba-tool domain provision --use-rfc2307 --interactive

It will ask you some questions, and here’s where we get into the “domain” naming philosophy again. Just make it the same as your DNS decision above. In my example, the Realm I chose was SAMBA4.MYDOMAIN.COM

Do do the upper-case! Why? I don’t know!

And for the “Domain” I chose “MYDOMAIN” (without the .COM). It’s pretty much like your workgroup setting, is all I can figure.

If you do it this way, then all machines joining your Active Directory “domain” will get the right DNS information for your DNS zone — because the AD server will only consider itself authoritative for SAMBA4.MYDOMAIN.COM and “higher”, but not for all of MYDOMAIN.COM itself — and it will forward those DNS requests on to your proper DNS server when it doesn’t know about them.

So be sure to set your DNS forwarder here to your real DNS server.

Cold, Cruel Kerberos

I’ve never know it to be so easy. I’m leaping with joy inside. Or maybe that’s lasagna.

cd /etc
cp krb5.conf krb5.conf.original
cp /usr/local/samba/share/setup/krb5.conf .

Then edit your new /etc/krb5.conf and change the REALM variable to the realm you chose: SAMBA4.MYDOMAIN.COM

I know! Can you believe it! It’s here where I feel a twinge of almost… non-sickness about MS. Ok it may even be stronger than that. A little.

Reboot again. Hahaha!

You Can Dance

Now, just start Samba 4 by typing in “samba”

It will give minimal info in /var/log/syslog – mine complained about CUPS not being there, but it wasn’t enough trauma for it to die, thankfully.

Now you’ll want to set up your administrator auth-y stuff, yes?

kinit administrator@SAMBA4.MYDOMAIN.COM
samba-tool user setexpiry administrator --noexpiry

Bad idea that no-expiry flag probably. But we’ve already established I’m naughty.

That’s about it! You can now fully administer it just like an Active Directory domain controller from Windows, using their remote server administration tools. Crazy, I know! That link is for Windows 8.1 download, BTW.

Also, the Samba website has a good howto on stuff like this.

The thing is, when you join a Windows machine into the “domain”, you have to make sure that you’re using your Samba 4 server as the DNS server for that machine, just like you would have to do with Microsoft’s Active Directory domain controllers. They need the filthy DNS injection.

Home Directories for Windows Users

If you want to have your Samba 4 server serve out home directories to your users, you accomplish that pretty easy. It just requires a “[home]” section in your smb.conf file.

That’s not a “[homes]” section like in Samba 3 by the way — just a singular “[home]”. It’s special. Apparently.

That section only requires a path and a not-read-only:

        path = /home/
        read only = no

You don’t really need local accounts for your users. Samba 4 will create crazy high-numbered fictional users and groups to service your Windows throngs. Just make sure that mountpoint has the acl and xattr flags.

Oh, and your administrator account will need the “SeDiskOperatorPrivilege” I think:

net rpc rights grant 'MYDOMAIN\Domain Admins' SeDiskOperatorPrivilege -Uadministrator

This will make it so that, if you use the Windows remote administration tools in Windows, you can create users that can have a drive automatically mapped to their Windows machine when they log in, and Samba 4 will create their home directory automatically.

The setup in Windows is a little convoluted. I’m no Windows person. But here’s a step by step that I followed and it worked great.

It should also be noted that the default setup seems to allow normal workgroup functioning to continue working as well. So even if you have Windows machines that aren’t the insanely more expensive “Pro” version of Windows, you can still map to the shares like you could to a workgroup.

But then again, that begs the question, why then bother with an Active Directory Domain Controller at all? Unless you want to spend a lot more money per seat on Windows.

Final Comments

I am impressed with Microsoft’s ability to impose a standardized way of implementing LDAP in conjunction with Kerberos. I am less impressed with their shameless violations of DNS to rope this in.

I haven’t tried it yet, but apparently you can pretty easily have your Linux boxes authenticate against Samba 4 as well. I think I may not be doing that. Well, maybe I will.

It is really nice and compelling that it’s all tied together. And it’s not so bad since Samba 4’s been able to bring it into the light. I’m undecided. It seems to work well.

Anyway, I hope this helped someone. I was very daunted by the whole Active Directory integration mess at first. But these Samba guys really have done a great job. I’ll be showing them some love. Of the monetary type! Well, I suppose unless…