Perl 6 Variable Types and Modules

I’m still learning a great deal about Perl and its typing. One of the things you hear a lot is that, in Perl 6 everything is an object. I’m finding this to be true, and it has some oftentimes surprising results.

For example, I’ve recently been playing with the DBIish module which affords database connectivity. We all know that working with date and time types with database columns can be a real pain, getting the formatting of the strings to insert correctly, and interpreting those when selecting out.

Perl has a module called DateTime built in. You can say it’s a module. You can say it’s a data type. You can say it’s an object. PostgreSQL has a timestamp with timezone data type (amongst others).

In my data model class, I defined an object attribute as a DateTime data type, just to play around and see what would happen.

class Shoe {
    has Pkey     $.id;
    has Str      $.description is rw;
    has DateTime $.created is rw;

    method retrieve(Pkey :$id = $.id) {
        $.created = %row<created_timestamp>;
        say "Created in the year {$.created.year}";
        say "warranty expires {$.created.later(:2years)}";

You don’t need to worry about creating a DateTime object, other than specifying that your class attribute will be a DateTime type. All the conversions to string, whether TO the database or FROM the database, get handled seamlessly.

Honestly, I didn’t expect this to work — but I’m very happy to never have to worry about date and time formats again, when dealing with database column types! 🙂

Perl 6 Lies – What Might Bite the Newly Arrived

Don’t misunderstand; Perl 6 is wonderful and I am having the best time playing with it – getting used to it beauty and strangeness. This article is to document some unexpected gotchas that got me along the way.

Optional signature parameters are sometimes not optional

One of the nice things you can do with Perl is create subroutines and class methods with parameter signatures that can take care of a lot of tedious coding for you. For example, suppose you want to create a method that changes a shoe in your database table. Your method signature might look like this:

class Shoe {

method update(Int :$id!      where { $id > 0 },
              Int :$color-id where { $color-id > 0 },
              Str :$name! ) {

... method code here ...

These are named parameters here, where you could call $shoe.update( id => 23, color-id => 8, name => ‘Hush Puppy’);

Parameters with a “!” are required. Named parameters lacking a “!” are optional, or you could put a “?” there. The “where” clause makes sure that whatever is entered will be constrained by the where clause.

Now, we have color-id being optional. However, if you leave it out, your program will bomb out with the following worthless error message:

Invocant requires an instance of type Int, but a type object was passed. Did you forget a .new?

This message goes away if you include the color-id parameter, however. So, this optional parameter actually is not optional.

But if you ask the Perl guys, they’ll say it is optional. It’s just that when using a “where” clause, a bit more is going on. If we didn’t have that where clause on color-id, it wouldn’t matter.

The issue is, Perl wants to use these where clauses in the signature to make sure that parameters are always constrained and typed correctly — and this includes parameters other than the one you’re checking. That is, your color-id parameter can also check stuff with your id or name parameters as well on invocation. So they have to bind early on, if we want to have that extra bit of wonderful functionality and consistency.

Normally you won’t have to worry about this. The only time it matters is when you’re using an optional parameter that includes a “where” clause in the signature. When you do that, the fix is easy, and makes good sense. In the case above, the color-id line would be changed to the following:

Int :$color-id where { not defined $color-id or $color-id > 0 }

The trick is just remembering: when using an optional parameter with a where clause, make sure the where clause will evaluate to true for the constraint — because that constraint will be checked even when the parameter is marked optional. And the documentation has been updated to make this somewhat more clear.

Using submethod BUILD and using attributes with class inheritance

This is very much like the same problem just described, when optional parameters need to be checked for existence when using the “where” clause in the signature.

Whenever you use a submethod BUILD in a class, you have full control over attributes that are assigned to that class. This also means that you have the responsibility of assigning them.

A great Perl shortcut is to have attributes automatically assigned by specifying them in the signature like this:

submethod BUILD(PosInt :$!id, :$!name, :$!address) {}

This lets attributes be set upon the .new invocation, just as normally happens, which is very handy. However, if you do something like use a custom type, like PosInt above, which expects integers greater than 0, you will get an error about trying to set $!id, even when you didn’t set it in .new(). It will complain about not meeting its type constraints, even though you didn’t set it to anything.

So, just like signatures using where clauses, if your variable is going to be undefined, you need to handle that in the body of the BUILD function and not in the signature — otherwise the auto-whatever assignment system will set the value to Any — which is not a PosInt. 😉 So don’t let that bit do the nice auto-assign stuff.

Once can happen twice, or thrice or more!

Perl has long been known for its powerful and creative control structures. One of my favorite new ones in Perl6 is the “once” control structure.

What’s neat is that it takes care of something we all often have to deal with manually. When you’re doing something in a loop, sometimes you’ll want to do things only once inside it, and after that once, though the loop continues, you won’t do that one-time thing again.

until $database.failure {
    once { say "Starting...";
           $log.writesys: "Starting..."; }

    ...all my normal stuff...

This will loop and do your bidding until $database.failure returns True (or whatever). However, the code in the “once” block will only be executed one time, no matter how many times the loop happens.

This very, very handy. But Perl 6 is also very multi-threaddy. So you cannot count on “once” to do something just once if you’re using multiple threads for your task. The variable state stuff just isn’t there to cope with such things.

The documentation for once is also updated with this information now, and yet here is another warning. 🙂

Parsing Command Line Arguments in Perl 6

Like many people, I do a lot of automation using Perl. Years ago I began making utilities more generalized than specific, so I could re-use them in slightly different situations.

This means using a lot of command line arguments passed to Perl so that it knows what to do. Always in Perl 5 I used Getopt::Long to parse command line parameters – a truly great module that was always there. Not so with Perl 6 – some very good command line argument parsing is built in.

The MAIN Subroutine

The key is discovering that Perl 6 supports the use of a subroutine named “MAIN” which is called whenever you run a Perl 6 program, much like a C program. The command line arguments can be captured and processed in the definition of the MAIN subroutine.

For example, say you were writing a utility to crop an image. You want to be able to call it like

cropimage --top=10 --bottom=15 --left=50 --right=30 picture.png

In Perl 6, this is all you would need to do:

#!/usr/bin/env perl6

sub MAIN ( $filename,
           :$left, :$right,
           :$top,  :$bottom ) {

    say "$left $right $top $bottom $filename";

Variables get automatically defined in the MAIN scope, so you don’t have to use “my”. And if you put a colon “:” before the variable name, Perl will treat that as a named parameter. Notice how $filename doesn’t have a “:”? So we don’t have to put a named parameter in the command line.  But the other variables have “:”‘s before their names, so that means they have to be a named parameter.

Worth noting as well that Perl seems to like having the non-named parameters come before the named ones in the sub MAIN definition. In other words, positional ones come first, then do named ones.

Optional and Required Parameters

If you want to make a parameter required, shout it and put an “!” after it’s name. If you want it to be optional, put a “?” after it:

sub MAIN ( :$left!, :$right?,  ....

would mean that the named parameter –left MUST be supplied or Perl will generate an error for the user, while the –right parameter is optional.

Parameter Default Values

Now, with our example, we may not need all of the parameters every time, so they should be optional. But we will also want to have default values of 0, since when the parameters are not specified, we’ll want to crop 0 pixels off in that direction.

So lets make all of our parameters optional and with a default value of 0. Of course, parameters with default values are always optional. It’s just a matter of setting an equality in the MAIN definition:

#!/usr/bin/env perl6

sub MAIN ( $filename,
           :$left = 0, :$right  = 0,
           :$top  = 0, :$bottom = 0 ) {

    say "$left $right $top $bottom $filename";

With this, we’ll get actual 0’s coming out in our “say” 😉 even if we pass in no parameters but $filename.

Constraining Parameters into Sanity

You know there will always be some smart aleck or some universal happenstance that will throw wrenches at you. It’s the curse of being our own form of order within other’s entropy. Take for example the guy who wants to crop 10.88242 pixels in on the left. Yeah, that guy.

This is where Perl’s variable type system can help. There are LOTS of types available. We’ll keep this simple. In our case, we don’t want to slice up our pretty monitor’s pixels into .88242 segments. We want whole pixels or nothing! An integer, in other words.

To constrain parameters to a type, just place the type name before the variable/parameter definition. You don’t have to, of course, but if you choose to, it can help sometimes.

#!/usr/bin/env perl6

sub MAIN ( Str $filename,
           Int :$left = 0, Int :$right  = 0,
           Int :$top  = 0, Int :$bottom = 0 ) {

    say "$left $right $top $bottom $filename";

But you know they’ll be right back, breaking it again, because they thought it would be cute to crop a negative amount of pixels from the image. So maybe you’d want to use the UInt type instead (Unsigned Integer) so the value will have to always be positive.

Another useful thing is specifying a type Bool for a parameter variable, which basically makes it into a flag.

Oh, did I mention you can use dashes “-” in variable names in Perl 6? That amazed me, along with the fact you can use full UTF8 charactersets.

So suppose we wanted to be able to specify an optional output filename for our cropped image.  We’d like something like this to work:

cropimage --left=25 --output-file=cropped.png filename.png

All we have to do is call the named parameter variable $output-file and make it optional.

#!/usr/bin/env perl6

sub MAIN ( Str $filename,
           Int :$left = 0, Int :$right  = 0,
           Int :$top  = 0, Int :$bottom = 0,
           Str :$output-file? ) {

    say "$left $right $top $bottom $filename";

    say "Writing " ~ ($output-file || $filename);

Bonus Error Messages!

If you tried any of these and got an error, you’ll see how nicely Perl formats these when you’re using the built-in parameter handling. It’s an actual detailed “Usage:” description that is given to the end user, detailing all the parameters and their types. Very Cadillac. 🙂

Installing Perl 6 on Debian or Ubuntu

Keep in mind, this is written on 09-Sep-2016 and stuff can change over time.

Perl 6 is developing rapidly, specifically, the MoarVM and Rakudo which comprise the environment that implements the Perl 6 specifications.

Every Linux distribution comes with Perl. Many have Perl 6 now as well. With Perl 6’s rapid development, these distribution packages of Perl 6 can be outdated.

This is how I installed Perl 6 in Ubuntu and this should also work for Debian. We rely on a system much like Perlbrew, called Rakudobrew to automatically download and compile the necessary stuff for us, and change our environment to be able to find perl6

Git Your Compile Tools

First, you need to be certain you have the tools and libraries necessary to compile stuff on your local machine.

apt-get install build-essential git

This will install the gcc compiler and various libraries along with the git version control system so we can download the latest rakudobrew and keep up to date over time.

Download and Install Rakudobrew

git clone ~/.rakudobrew
 echo 'export PATH=~/.rakudobrew/bin:$PATH' >> ~/.bashrc
 source ~/.bashrc

The first line uses git to download the rakudobrew archive into your home directory in .rakudobrew. No biggie. These instructions are taken (mostly) directly from the Rakudo site.

The second line alters your shell environment PATH to include this directory’s “bin” subdirectory. This is wildly dangerous unless you trust absolutely where you are downloading from, which is never a good idea. You can always just compile it yourself if you follow the instructions on the above-linked Rakudo site.

The third line alters your currently-existing PATH in your terminal to add that .rakudobrew/bin directory (you could also close your terminal and open a new one instead).

Have Rakudobrew Compile Rakudo and MoarVM

This bit’s the easy part now, thanks to Rakudobrew.

rakudobrew build moar
rakudobrew build panda

The second line builds and installs Panda, which seems to be a cpanm-like thang for Perl 6 modules. To have a complete Perl 6 experience, you’ll need some of those modules that are commonly bundled up with Perl. You can get them with

panda install Task::Star

And they’re you’ll have Rakudo Star! Like cpan or cpanm, it will take a while to download and compile the modules.

How to Run a Perl 6 Program

Easy enough if you’ve done the bit above to include .rakudobrew/bin in your shell path. You can just type “perl6” and have an interactive interpreter to play around with.

Or if you prefer to edit a file, you can run the file with

perl6 filename

Tough, eh? Or if you’d rather the OS knows to do perl6 for you, give it that magic hashbang at the top of the file like so

#!/usr/bin/env perl6
my $code = "about normal";say "I know nothing " ~ $code;
("I know nothing" ~ $code).say;
say "The new $code";

No need for warnings. No need for use Modern::Perl . No need for strict, Try::Tiny, etc.

There ya go!

Perl 6 First Impressions

Perl Camelia LogoStarting at the beginning in Perl6, I had no clue. I only have a slight one now. But I’ll be documenting some of the things I learn in subsequent posts, in case it can help anyone in the future.

This first post will be about initial impressions, coming from someone who knows a bit about earlier Perls, and has held off actually putting Perl 6 into work.

Background Info

Perl now runs via a VM – right now either the standard JVM or a new one called MoarVM (which people seem to like best and is default, and which I like too because it hasn’t hurt me yet).

Perl itself is a language specification. It relies on the Rakudo compiler to implement that language specification and tie it into MoarVM. This is a great separation of concerns — MoarVM can worry about interacting with the operating system, Rakudo can worry about what it needs to tell MoarVM to do and what the heck a Perl coder meant to happen, while Perl itself remains only in the abstract.

Why Use Perl6 when Other Languages Work Great?

For me it was curiosity. I love coding in Perl, for the most part. It’s the most flexible and powerful language I’ve encountered. I’ve been curious about Perl6 for years, but resisted putting any of it into production until recently. There’s something about it now that feels solid and ready. Here are some of the things that feel compelling to me right now.

Close to the OS

One of the several reasons I reach for Perl before any other language is because it runs close to the OS. Perl grew out of shells and shells grew out of Perl. I was worried I might lose this with Perl6. So far, I am very happily surprised. There is much that makes interacting with the shell/OS even easier – things that don’t require finding the module du jour and reading or refreshing up on it. It just flows in and out of the language.

Bondage or Freedom

It’s your choice! Perl has traditionally been very flexible, and many people who like to be told exactly what to do, or like to tell others exactly what to do — these people become easily exasperated by the freedom of expression Perl offers.

Eventually Perl modules were created that acted like tight girdles and cat-o-nine-tails upon those people, to help satisfy them. Now Perl6 has its own build-in dungeon basement!

You don’t have to use it though. But it’s there if you want it, and it’s seamlessly part of everything.

For example, you can enforce variable types if you want to, and a great many of them too, in bizarrely interesting ways. And you can coerce them around. Or you can leave everything all free-form.

You can use positional parameters when calling methods or subroutines, enforce types there too, or use named parameters, or just sloppily slurp in data. (or neatly slurp 😉 )

And there are many ways to die, even in an orderly way with exceptions. And the default error messages are pretty great, most of the time, with calling stacks included by default, without having to use modules.

And control structures… well, there’s a veritable zoo. Or just your standard, well-known tried and true for those who shrink at variety.

Perl is Smart

Perl has never been limited to being discreetly-sized Lego blocks that kids can easily make a toy house from. It can be used that way, of course. And often it is, by people new to the language.

For me, Perl’s greatest strength is that it has always allowed me to work very intimately with the structure, flow, manipulation and presentation of data, in whatever form, as an actual capability built into the language itself — and to accomplish it in ways that are both optimal and intuitive.

These capabilities have only increased with Perl6 as far as I have seen, though some semantics for getting there have changed. But those changes I believe are justified because I can see how they make sense, to give room to other capabilities that have been added, and to provide some clarifications.

In case that doesn’t make sense, one little example is that string concatenation is no longer done with a dot “.”. In Perl6, variables (objects) can have a dot after them to call a method on that object — that could potentially get confused with a concatenation dot. So they’ve made the concatenation into a tilde “~” instead. It actually looks better I think, but it was a nightmare to figure out when first starting to play. I’ll be documenting some more gotchas as I run across them.

BTW – that last change mentioned also influences other things. For example, “$string .= $thing” has become “$string ~= $thing” for concatenation.

What Sucks about Perl 6

Oh my god everything! (I thought at first). I really did. And I kept running into problems and errors that naturally were due to Perl6’s newness breaking it, and not my own ignorance…. but of course, like most ignorance, it doesn’t recognize itself — and it turns out that every time I thought I had found a bug in Perl6, it was really me not understanding how something had changed, and why.

I may run into legitimate bugs at some point though! I’m just now starting to really dive in. But they’ve had a very long time to work through things.

But it does suck that there are odd little differences between Perl 5 and Perl 6 here and there that can cause problems. But that’s the nature of change. So far nothing sucks really, once I move aside my own irritation at having to learn a few new things.

What’s Surprising about Perl 6

What surprises me most about Perl 6 is that Larry and the team have managed to preserve so much of what was truly monumentally great about Perl 5 and bring that into a context that is offering so many conveniences that were previously only attainable by using sometimes kludgey modules.  For example, I love how we handle command line arguments now. I’ll get into that. No more real need for stuff like Getopt::Long for most things.

Also I had heard that Perl 6 takes a long time to start up and actually run something. This is not the case, at least any more. It feels pretty instantaneous so far, and considering that it has the functionality of Moose built in, that’s awesome! 🙂

I haven’t done any performance metrics on it, and don’t really plan to.  Well, maybe. But it has been a lot more fun than I imagined, and I’m actually looking forward to stopping writing this and getting back to more using it.

Another thing that is surprising to me is its handling of numerics — the math is damn good, and you don’t get weird rounding errors from what I’ve seen. Also it has these magical ranges of numbers that can go clear up to infinity, or skip in defined sequences, and has even type testing built in like is this number a prime… ?

What’s surprising is that it feels like it’s a whole magical world again, and one where you can use convention, or establish your own conventions just as easily.

Anyway, I’ll leave this at that. I’ve really only taken the first little test drive, and not put it through an real industrial rigors. That will be coming as I gain deeper familiarity with Perl 6 — which I have decided now, I intend to do.

Perl Major Mode for Emacs – How to Install

If you are an emacs user and the normal Perl mode is not formatting your newfangled Perl6 code properly, you may need a different mode than perl-mode.

Thanks to Hinrik Örn Sigurðsson you can get an Emacs major mode called perl6-mode from Github. And if you’re lazy like me, you’ll probably just want to have Emacs install and compile it for you from the MELPA archive.

To do that, in your .emacs file put:

(require 'package)
(add-to-list 'package-archives '("melpa" . ""))

This will allow you to download emacs packages/modes from the MELPA archive for Emacs. Then in Emacs do a:

M-x package-refresh-contents
M-x package-install

From that point on, emacs should recognize Perl6 by file extensions or even if you have the magic headers like #!/usr/bin/env perl6 at the top of your programs.

So far, I’ve found Hinrik’s perl6-mode to be a godsend — I was going nuts living with the perl-mode getting confused. The only thing I’ve noticed is that it doesn’t colorize pod stuff the way I’m used to.

Thank you Hinrik!!!

Fixing the Reckless Debian Security Update to Samba 4.2

If you’re like me, and were wanting to get the Samba security fixes in place quickly for Debian Jessie version of SAMBA — you’ll maybe have learned that they decided not to backport the fixes for man-in-the-middle, but instead upgrade the whole thing to SAMBA 4.2.

That’s all well and good — but the security update broke SAMBA on my Debian boxes in the test lab (which run Jessie).

The fix was pretty simple after some fairly serious wasted time Googling for it. By default, it SAMBA will now require the winbind package to be installed, otherwise it will silently fail to start.

The symptom I had was that internal DNS resolution for SAMBA stopped working.  And nothing was bound to port 53.

Of course, that was because Samba wasn’t starting at all.

I hope that helps someone — just make sure winbind is installed and running.

They really should have tested this better, even with such a bad flaw in the Microsoft and Samba stuff.

Pipe to Perl for the Quick and Dirty Big Jobs

I always find myself using Perl for systems administration tasks, particularly when you must do things on many systems at once. It is in Perl’s DNA to be down in the trenches of the operating system.

For example, last night I had an NTP configuration file I needed to get out to a few dozen machines. I had a list of them from a display of managed machines, and they were tabulated like this:

62 8 Workstation 6 Company, Inc.
61 8 Workstation 6 Company, Inc.
63 8 Workstation 6 Company, Inc.

And I need to get an NTP configuration file to each. Sure, you can write a shell script using sed to do it just as easy, but with Perl you do get a little more flexibility to do the strange things that must be done from time to time.

#!/usr/bin/env perl

foreach (<STDIN>) {
    my($id, $host, $ignore) = split;
    print `scp -v ntp.conf root\@$host:/etc`;

Of course, if you like to live dangerously, you can always put in full ssh commands. You can make the command you want to run be an argument. You can use any of the numerous Perl modules available to handle specific things or concerns.

The command for this is something like:

list_hosts | my_perl_program

Perl just lets you read from STDIN as a normal pipe, with nothing extra you have to do. Columnar/tabular data – it’s so easy. Calling a shell command and sending the output anywhere? Right there.

Edit that to reload your NTP daemon on each. Or make the command an argument when you call the program. Or an option, using Getopt::Long, or whatever you like.

Anyway, my suggestion here, start using some Perl. For admin purposes, it’s like the shell, on multi-dimensional steroids that don’t cause cancer.

Creating a Debian GNU/Linux Server – a Minimal Install

When you’re creating servers you usually don’t want the overhead of a GUI. This is particularly true when you’re creating a virtual server. You only want the stuff you need. And the basic Debian, of course, to hold it.

Grab an Install Image

Debian has several install images available. I usually prefer the network install image (netinst). It’s minimal in size, giving you what  you need to install, and then downloads only what you need.

I usually use the 64-bit one as well, even though you can save a little memory using a 32-bit one. There is a “Live” one, but I’ve not had good luck with it in unusual circumstances, which is often the case when I’m making servers or little devices.

If you won’t have a working network connection, get one of the full images instead. All Debian images are updated from time to time, as updated releases happen, so check occasionally.

Install Media

If you’re installing into a virtual machine, you’re already set with this ISO image. If you’re installing onto hardware, I find a USB thumb drive is most useful.

The Debian ISO images have long been set up to create proper bootable disks by just dumping the ISO data directly to the USB.

To create the USB stick in Linux, it’s just a matter of using dd to dump it. In Windows you’ll need a program such as Unetbootin to create the bootable USB stick from that ISO image. Unetbootin also runs in Linux, and it often packaged in Debian already.

But the quick and easy way is just use a command prompt. Find your USB drive’s device name – that’s the hardest part. In Gnome you can look at disk utility, or tail the syslog and watch while you plug it in. 😉

$ wget

$ dd if=debian-7.6.0-amd64-netinst.iso of=/dev/usbdrivename bs=4M ; sync

Please be very careful with that dd command. You can easily destroy your system or data if you put in the wrong of= parameter. But if you don’t destroy your system or data, we’re ready to go.

Make sure you set your BIOS or UEFI to boot from the USB drive, or if you’re installing into a virtual machine, don’t forget to attach it as a CDROM.

The Install

The first screen you’ll come to when booting will let you pick between a normal install and a graphical install (along with some advanced options, which we’ll hopefully cover later).

I usually pick the normal install because you don’t have to worry about moving a mouse around as much, nor do you have to worry about there being any strangeness with your graphics cards and this particular kernel (which is rare any more on a base install).

The next screen lets you pick the language, which defaults to English, your location, your keyboard layout (all English and American by default).

After just those few very basic questions, the install will start loading some kernel modules it might need, including those needed to activate your network interface.

The Network Interface

The Debian installer will now try to set up any network interfaces it finds. It will try to configure both IPv6 and IPv4 with DHCP. You can, of course, specify the configuration manually if it doesn’t detect network configurations.

If you’ve got a good hostname, it will fill that in from DNS automatically, or you can specify it yourself. Same with domain name.

If no hardware interfaces are detected, you can always go back to the main menu and try loading kernel modules manually, if there are any, or just weep. It seems that just about everything is recognized any more.

User Account Setup

Next you’ll be prompted to enter a password for the root user. Debian enabled root logins by default. If you want to disallow them, you’ll need to install “sudo” and  add a normal user to the “sudo” group, and disable the password of root (passwd –lock root). If you do that, you can still log in as root with SSH keys, if you have them.

After you set root’s password, you’ll be prompted to add a new, normal system user. I hate that they added this bit. But just enter something I suppose.

Final Little Miscellaneous Config

You’ll need to enter in your timezone. I know! So hard.

Disk Partitioning

Gods help you. There are so many ways you can do it. If you have one drive and want everything just on that drive, select “Guided – use entire disk”.

I never do that, though, unless it’s a virtual server. I like to set disks up with LVM, and RAID multiple disks together. You can do all that nice and easily in the disk partitioner if you like. It’s very well laid out. But more complex disk partitioning schemes is beyond the scope of this quickie walk-though.

You can, of course, read through an intro to various Logical Volume Manager disk stuff if you like. The Debian installer automates a lot of that.

Another nice feature is that you can set up full disk or partition encryption here, too, which is convenient for your secret purposes. You can encrypt a partition or a RAID device or LVM volumes themselves. Many people argue over which is best to do. I have my moods, and like variety.

I rarely choose anything but to place all files in one partition. This is because I would rather muck about with the underlying volumes and and devices. However, there are some very good reasons to separate your mountpoint concerns in some situations, particularly with read-only and highly space-constrained systems.

After answer the partitioner’s few questions, you’ll be presented with its final understanding of what you wanted, which you should verify. Let’s look at one I just did:

Debian Partitioner

Here you see I chose to put everything in one partition using LVM underneath it. Which, of course, the installer lies about, and really creates a boot partition for you as well.  And even a UEFI one if your system’s got that. It’s good practice to keep a separate boot partition.

The Debian installer organizes things from the most abstract up top, to the most low-level stuff below. You’ll see the LVM volume group and logical volumes it wants to create on top, and the actual disks that contain those down below.

Generally, when you want to make changes, start at the top and work your way down, if you really need to. I’m going to skip the LVM configurator and the RAID configurator, because we could really just go on forever with that.

The point is, the Debian installer gives you incredibly good access to setting up your storage in all kinds of ways. They were the first to support so many different things at boot, and although it can bring complexity, you don’t have to deal with that complexity any more if you just follow their defaults (all in one place).

So… after it looks right to you, or doesn’t and you’re just trusting, you select “Finish partitioning and write changes to disk” and you’re good to go.

Oh, you can move the cursor to any of those top mount points and change the filesystems you want to use, or what gets mounted where, all that stuff. If you like, or have the need. Even mount options that will get automatically passed along to your system’s fstab

You should know that the base Debian system is very small indeed. The 8Gb you see here will leave plenty of room for many, many things that servers do. The end user data stuff, probably not. But you can always tack on a new disk for that, or partition it in a different place.

Installing the Debian System

After your partitioning and formatting are successful, the installer will install the base Debian system, which is, well, the base system. All you need to boot up and get going, adding whatever packages you like.

After it does that, it will want to set up updates for you, and will ask where you are so that it can find a Debian archive mirror that is close to you.

Pick one! Then it will ask you if you use an HTTP proxy to get out to the net. If you do, enter the info here. And as an aside, setting up a proxy for Debian packages is really, really nice when you do a lot of virtual machines. It saves you having to download the packages to every machine, which can save a great deal of time, over time. apt-cacher-ng is my personal favorite.

It will then download archive package indexes, install and update any packages that may need doing, and then it wants to know just what type of Debian system do you want here?

Software Selection

If you’re doing a server, uncheck the Debian desktop environment. It’s too fat. Odds are you don’t want a print server, either. Why is that checked by default? Whose version of madness is allowed to permeates us all so?

Standard system utilities is all you want. And SSH server.

But, maybe you’ll want to run this as a DNS server, whether master or slave? You can check that and it will set up Bind9 for you.

File server will give you Samba 3 and some utilities. Mail server gives you Exim 4 – which I finally, after years of clinging to Sendmail, surrendered to myself. Web server gives you Apache 2

But all those others can be easily added with apt-get later. Standard system and SSH is the way to go. Check that, and away you go.

Boot Loader

After the install finishes, the installer will want to place a boot loader on the drives. The boot loader used by default is Grub. It’s pretty good at detecting other installed operating systems, which hopefully you haven’t overwritten by telling the installer to use an entire hard drive. Well, unless you want that.

The installer seems to do a very good job of working with UEFI as well. However, I have noticed that from time to time, Windows 8 will not like to boot after installing a new boot loader. Be warned if you’re trying to dual boot with Windows 8. Sometimes it works, and sometimes it doesn’t, and the only thing I can think of is different motherboard manufacturers implementing UEFI differently.

Let the boot loader install, and your new minimal Debian system should reboot up just fine, ready for you to begin work on making that new server.

The Debian developers have done a really great job of creating a very minimalist and simple installation system. And for popping out one virtual machine after another, it really can’t be beat.

Add a Simple Samba File Server as a Domain Member

ssambaIf you already have an Active Directory Domain Controller in place, diligently servicing all your needs and making itself indispensable, hopefully you’ve chosen Linux and Samba 4 to fulfill this.

If you haven’t used this free and open version of Active Directory and the domain controller, perhaps you’d like to? For the latest and greatest Samba 4 version that you compile yourself, you can follow the steps outlined for Debian Wheezy (and possibly Ubuntu).

Remember, too, that Debian backports now has a much more recent version of Samba 4 in its archives, and it seems to be working great right now.

But suppose you have that beast in place, and you want to add, say, a network file server — some storage that can be accessed by all domain users and whose rights and permissions are determined by your Active Directory domain controller.

It’s easy to just create a share on your Samba 4 Active Directory Domain Controller, and serve that out, with all the permission “goodness”.  But maybe you don’t have tons of storage in that box, or in that VM, and you don’t want to, either.

Or, you could find some filthy user’s computer in the domain that has a lot of hard drive space in an array, and map it out to the others. (I mean lovely user – it’s a joke!#$@!). But we know that’s a terribly unsettling idea.

So why not build out or spin up a new Samba 4 server that’s pretty much dedicated to housing user data, whether that’s shared between the masses for apps or data, or just for daily backups.

I’ll tell you why. Because you’re frightened. How can you get one Samba server to listen to another Samba server and believe all its tales about the users and their permissions? How will the UIDs and GIDs match up for filesystem stuff? Sure, you know it’s possible. You’ve looked around. You’ve seen all kinds of insanity for linking Samba 3 into Windows AD/DC, mapping local users and huge cut-‘n-paste swaths of wild configuration blocks.

But if you go Samba 4 again, will you find any documentation that is currently to the point where Samba 4 development is? Or to the way that Debian has mutated it?

The answer is yes! All over the place, including the Samba site itself, and it’s all whacko and incomplete, if all you want to do is create a simple file server – you just want a simple Samba 4 file server that is a domain member file server. You don’t care if users can ever log into the Linux box. You just want them to have access to Windows file shares served with Windows users permissions and rights honored.

And you may think… I’ll just make a big RAID disk on some server, and serve out to my One Box To Rule Them All Samba 4 AD/DC – and then it has big drives and plenty of space! Well, it would if NFS or CIFS handled extended filesystem attributes… Alas! We are thwarted.

So here’s what I did:


You must a Samba 4 Active Directory Domain Controller running just fine already. Or a normal Windows-y one if you must run a Windows one for some nonsense reason.

You must have a Debian (or possibly Ubuntu) server ready to go with only the minimal stuff installed – like the SSH server. That’s because it’s proper. And you will be proper. Another distribution is fine of course, just don’t complain to me.

Don’t use Debian Wheezy’s version of Samba 4. It’s not ripe yet. And don’t use Samba 3. It’s overripe. (Simply too many notes) Either roll your own Samba 4 from source or enable the Debian backports repository and go with that version of Samba 4, there.

If you follow the instructions to roll your own, skip the samba-tool domain-provision step and the Cold, Cruel Kerberos sections!It makes your new Samba 4 server have delusions of grandeur and it won’t want to listen to your already-existing AD/DC.

If you go with Debian backports, this is what you’ll need:

# apt-get -t wheezy-backports install samba samba-doc samba-testsuite winbind libnss-winbind
# apt-get install acl

Setting Up the Samba 4 Domain Member

The smb.conf [global] section

If you’re using the Debian version of the /etc/samba/smb.conf from backports, throw away everything in it, because it’s garbage Samba 3 stuff and they haven’t bothered tidying anything up.

For a simple Samba file server, you just need your [global] section and your share definition. I’ll highlight some of the stuff in it, after this example listing (that works just great). Be certain that your filesystem that serves out the stuff is mounted to support xattr and acl’s. (in your /etc/fstab put the mount options “user_xattr,acl” in place of “defaults”).

Anyway, here’s that /etc/samba/smb.conf file:

  netbios name = <servername>
  workgroup = <win domain>
  security = ADS
  realm = <kerberos realm>
  encrypt passwords = yes

  idmap config *:backend = tdb
  idmap config *:range = 70001-80000
  idmap config <win domain>:backend = ad
  idmap config <win domain>:schema_mode = rfc2307
  idmap config <win domain>:range = 3000000-4000000

  winbind nss info = rfc2307
  winbind trusted domains only = no
  winbind use default domain = yes
  winbind enum users = yes
  winbind enum groups = yes

  vfs objects = acl_xattr
  map acl inherit = Yes
  store dos attributes = Yes

That’s it, really. I love Samba 4 for this. Consider me gushing and oozing again, all over the Samba development team again for doing such a damn fine job pulling so much together. Very well done.

Here’s what some of those things are:

  • netbios name – anything you like – I like to use the hostname portion from DNS
  • workgroup – that’s your Windows “domain”. Make it the same as your main Samba 4 server’s.
  • realm – your Kerberos realm – again, make it the same as what your main Samba 4 server’s is.

Now, the idmap stuff should be fine for you to use, if your AD/DC is standard Samba 4. The range values are UID values that map to local users (the asterisks) and your AD users (the <win domain> ones).

By default, at least right now, Samba 4 is by default using the UID range of 3000000 – 4000000. That can be changed when you set up your Samba 4 AD/DC. A Windows-based AD/DC is probably different. Find it out, and put it there.

The local mapping range just shouldn’t overlap your domain range ones. And I kept mine well above any UIDs I’d really be using locally on this machine (in the 70000’s… 😉 )

I hate winbind. I just copy and pasted that bit from some Samba site documentation dealing with domain member servers. It seems necessary, especially the nss bit for letting your system know about domain users on the AD/DC.

The last 3 lines are critical if you want permissions to be right on the filesystem here.

The smb.conf Share Definition

Behold the glory of letting a fascist AD/DC handle all your decisions for you!

   path = /srv/storage
   read only = no
   admin users = "@MYDOMAIN\Domain Admins"

Whatever path you want there, Sparky. And the MYDOMAIN bit is your domain of course, just like above. And oddly, this is the tricky one that caused me quite the headache.

After I got all this set up, I couldn’t set permissions on the share as a domain administrator. Permission denied. You have to add the “admin users” parameter even for domain administrators. I like this because it gives your local root the final authority still. But nobody mentioned this and it took forever to figure out. Here, I have the admin users be anyone @MYDOMAIN who is part of the Domain Admins group on the AD/DC.

But I digress.

The Final Steps

Of course, reload or restart this Samba daemon and its associated daemon cohorts of nmb and winbind.

# service samba restart
# service winbind restart

I hope you didn’t “provision” this Samba 4 server with samba-tool…

Also, remember to edit your /etc/resolv.conf file to make sure the nameserver you’re using is your Active Directory Domain Controller. If you don’t, you won’t be able to join the domain, since it relies on that absurd DNS injection flotsam.

The next step is to join your domain as a domain member.

# net ads join -Uadministrator

No, it’s not magic. It just gets your domain info from you smb.conf file.

If you got an error, you might want to make sure that you’re using your main AD/DC server as your DNS server in /etc/resolv.conf – that is, make sure your ‘nameserver’ is set to the IP address of your primary AD/DC. If it’s not, you won’t get the revolting Microsoft DNS injection they made necessary, and you’ll fail.

Remember the NIS/YP stuff? No? Well, then just do what I say:

# edit /etc/nsswitch.conf

Then add “winbind” to the end of the passwd and group lines, while leaving the rest of the file alone:

passwd:         compat winbind
group:          compat winbind

Final Thoughts

You might want to reboot. Make sure your mountpoint for the share you’re offering has those attributes specified in /etc/fstab.

You’ll probably want to grant your domain admins the ability to control files and permissions on your shares as well:

# net rpc rights grant 'MYDOMAIN\Domain Admins' SeDiskOperatorPrivilege -Uadministrator

But maybe not! I think it’s pretty well handled by specify the admin on the share level of the smb.conf file. But I did both, because I was flailing with that problem I mentioned earlier, and will not reiterate out shame.

Anyway, that’s about it. Except for, of course, always be sure you install ntp since time is so critical for this implementation with Kerberos. Just a few steps really, but lots of words and background info on why. Hope it helps.