Hardware Hump Day: Cloning your Raspberry Pi
Do you ever need to make a lot of Pi? This trick will save you some time!
I'll be the first to admit I was a latecomer to the Raspberry Pi party, but now that I'm in I have to say, it's one of my favorite pieces of hardware. It's a tiny Linux computer with HDMI out and network capability; what's not to like about it? Well, there is one downside to all that functionality: Configuration can be a pain. If your project only involves one Raspberry Pi, it's honestly not so bad. But things balloon very quickly when you need to have three or more Raspberry Pi boards all running the same software.
Of course you can boot each board, install your software from a thumb drive or online repository and then move on to the next. If you're a little more clever, you might write a script to install your software and set your configurations so you don't have to do it manually. But if your configuration is particularly large or complicated, this might take a very long time. So what's the answer? Simple: Just set up your first Raspberry Pi and then clone the SD card onto the others!
Alright, it's not exactly simple. You can't just copy all the files from your Raspberry Pi SD card and paste them onto another, because there are actually multiple partitions on the card that need to be present for the Pi to boot correctly. Luckily, there is a method for copying both the data and the structure of the card: Disk Imaging. When you create a disk image using a tool like Win32DiskImager, you make an exact copy of the entire storage device, including the unallocated space on the drive. That last bit, unfortunately, is going to make things complicated. Since the disk image will be the same size as the disk that it was imaged from, and since no two SD cards are exactly the same size, the image of one "16GB" card may not fit on another "16GB" card. Also, do you really want to hang on to a 16GB disk image that's mostly unallocated and unused storage space?
So we do need to make an image of our "fully-configured" SD card, and then we'll have to shrink it. But before we make an image, let's make sure that the partitions are as small as they can be and organized properly. Chances are, when you first booted your Raspberry Pi, it asked if you wanted to expand the filesystem and you said yes, which is fine! However, now you're gonna have to remove all of that free space from the Linux partition. The best tool for this job is GParted. If you have access to a Linux machine, you can run GParted there, but if you want to use it on another machine you can also run it from a live USB stick. This process will look slightly different depending on your source SD card, so I'm not going to cover it here, but most of the time it's as simple as clicking and dragging. You can read more about it here!
So now that we've shrunk the free space out of our partition table, let's make an image. (I'm going to assume you're doing all of this in a Windows environment, which I understand isn't a great assumption, but many of the tools have equivalents on other platforms.) There are a lot of disk imaging tools out there, but for Windows, I think the simplest and best is Win32DiskImager. Once you've installed and launched the program, you'll be presented with a very simple interface:
To make an image, first give your image a name. Make sure you select a destination with enough storage capacity for a very large disk image. Now, make sure your source SD card is inserted in the PC and select it from the "Device" menu. It is super important that you double and triple check you've selected the correct device. If you fail to select the correct device, you will cause irreversible damage to whichever device you select. But don't take my word for it:
Once you've made sure you have the right device selected, go ahead and press "yes." Once you've stared at this for long enough:
...you'll have an exact image of your source SD card sitting in your destination folder. The temptation at this point is to insert another SD card and press "write," but that won't work yet. First, we need to trim all of that unallocated space off the end of our image. Once again, Linux has the best disk management tools so we're going to use fdisk and truncate. Since I'm on a Windows machine, I'm going to use these tools under Cygwin! Cygwin is a Linux-like environment for Windows with many of the commands and tools that are familiar to Linux users. If you want to manipulate your disk image under Cygwin, you'll need to move it to your Cygwin home folder because Cygwin can't "see" most of your Windows file structure.
In the Cygwin console, the first thing I like to do is toss a few whereis commands:
Now I know the paths for each of these tools so I can go ahead and use fdisk to look at where my partition table ends and then use truncate to chop off the remainder of the image. First I'll show you what that looks like, then I'll explain what we did.
Looking at the above table, you can see all the separate partitions in the image that I was talking about. You can also see the "Start" and "End" sectors of each partition listed out. If we look at the end sector of the last partition, it's about 16 million. If you multiply that out by 512 bytes per sector, that's only half the capacity of our 16GB drive! The rest of the image is unallocated and essentially wasted space. To trim that wasted space from the end of the image, you can see I've written a truncate command on the following line. For clarity's sake, here it is again:
/usr/bin/truncate --size=$[(16861183+1)*512] Roshamglo_pi.img
What truncate is going to do is remove the last n-many bytes from our image file (in this case Roshamglo_pi.img). We want to trim everything back to the ending byte of the last partition, so we set the size to the end sector of the last partition plus one (because sectors are zero indexed) times 512 bytes-per-sector.
After running this command, go check the size of your disk image and you should find it greatly reduced. Now you can put it on as many SD cards as you want. Load up Win32DiskImager one more time and select the freshly trimmed disk image. Insert a fresh SD card, select it in the Device menu and click "write" to create your first clone.
Congratulations, now you can make hundreds of copies of your pre-configured Raspberry Pi SD card! If you have any questions about this process or know of a better tool or process, let's talk in the comment section! Now go forth and create a CLONE ARMY!!!
Pay close attention to Nick when he says you can screw up your hard drive if you're not careful. Just a few days ago I was cloning a Pi card and didn't pay attention to the drive letter (and warning!) Win32DiskImager gave me. I wrote an 8GB image onto a 1TB external drive, instantly destroying the file system already present. I'm currently running a wide variety of tools to get back what data I can. With great power comes great responsibility...
After truncate, image still burns to SD card but raspberry pi don't load. Can you please assist.
Linux admin senses are tingling, couple of tidbits to cover when doing this method:
Make sure to change ssh host keys per PI, this makes it clear if somehow ssh access is attempted to the wrong PI. Odds are 1 to 999, but still.
Make sure the distribution used accepts dynamic MAC addresses for interfaces, ie a new MAC gets eth1 instead of old MAC's eth0. This may not be noticed if DHCP is attempted for any interface by the distro, but certainly if assigning static addresses (but not by DHCP/MAC).
Finally, it's Linux, have fun. :)
For us Windows users, how can we see the windows file system to access the RPi images when using the Cygwin terminal? My images are in D:\Documents\RaspberryPi\OS-Images.
Never mind, the df command shows how the disks are mounted.
I'm not sure how this got attached to another reply using my member number, so I'll try to put it here again.
There is a MUCH easier way to do all this - piclone. Comes with the full Rasbian. Check it out.
Two things...
First, SD cards should always be formatted with the SD Association’s SD Card Formatter, and never anything else, including every tool bundled with your operating system. I’m not as familiar with Linux systems; as long as gparted doesn’t modify the formatting on the SD card itself, that should be fine, but if it does, using it isn’t the right solution to this problem.
Second, you really should have written this post around Etcher, a modern, cross-platform solution for burning images to SD cards, rather than Win32DiskImager, a single-platform solution. If you really have to upload new disk images to a whole slew of Raspberry Pis, then you might have to do so as a team of people, each using their own equipment, rather than as a single lonely Windows user.
I may be single, but I am not lonely!
-A Windows user
That's true if you're creating standard SD cards that can be read on any system. But SD cards used with Linux, including on the Pi, often contain file systems other than the ones that are part of the SD standard (which are FAT, FAT32, and exFAT). You can't create those non-FAT file systems with the official formatting software from the SD Association.
The fact that exFAT is used for cards larger than 32GB is the reason that many systems don't officially support SDXC cards. The licensing for SDXC is costly... but not because the SDXC license itself is costly, but because support of exFAT is mandatory and requires a non-free license from Microsoft. The license for SD (any format) is $1,000 per year, or free if you only implement SPI mode. (The SD readers on Arduino shields are SPI-only and thus avoid the cost of the license.) The hardware in the Pi (at least from the B+ on, I haven't tried the original version) actually works just fine with large SDXC cards, but it can't claim SDXC support because of the lack of an exFAT license.
If you HAVE created multiple partitions on your SD card that include non-FAT file systems, you can't reformat the card back to its full capacity with the official formatter. Sometimes the only way I've been able to do it is to use DD to write zeroes to the card (or at least the beginning of it) to wipe out the partition table. After that, you can use any software (including the official formatter) to create partitions on it normally.
I have been using the SD Formatter all the time to wipe RasPi cards for use in my next project. It will show available space of like 100mbs, then afterwards show 16gbs or 32gbs. Not sure why you said it doesn't work that way.