Open Source ALL the Things!
SparkFun is committed to open source - not just in terms of hardware, but content and software as well.
For a time now SparkFun has been making a name for itself as a player in the Open Source Hardware game. We helped draft OSHW Version 1.0 and are regular contributors at the Open Source Hardware Summit in New York. This is a point of pride for us. It's something we truly believe in and that's evidenced by our growing line of OSHW products (embedded below: an incomplete list of current OSHW products) and regular posts about what's happening in this burgeoning world.
Today, though, I want to talk about what else we can open source. It turns out there's quite a bit!
Let's start with images. One of the things that helped SparkFun stand out in its early days was crisp, high resolution images of all of our products from multiple angles. As a company built by engineers we know how important it is for one to have a clear mental picture of any new component before the credit card comes out.
Look at this swell image. Look at it!
We've now licensed all of our images under Creative Commons Attribution-NonCommercial-ShareAlike 3.0 (CC BY-NC-SA 3.0). Basically we want you to freely use our images in your projects or for whatever reasons you decide to use them so long as you provide attribution and aren't using them for commercial purposes (like that Chinese website that ripped off all of our images last year). Being realists we can definitely imagine scenarios where a commercial use of our images is totally cool so you can still contact us to work things out.
So we have hardware and images but something is still missing. Something big... the elephant in the room. Software.
Software is where open source began. It's why we call it open source... you get the source code and do what you want with it. Is SparkFun big into open source software? Yes, internally. Externally, we're on our way.
Our IT department is almost a 100% Linux shop. The website you're looking at runs on a stack of servers primarily running the Debian flavor of Linux with MySQL (an open source database layer), PHP (an open source scripting language), and Nginx (an open source web server layer). You could call it a LNMP stack. A handful of our internal software solutions are open source but there's certainly a mix of proprietary packages in there out of necessity and function.
Our primary internal software package, however, is one of custom origin called Sparkle. Sparkle is a sprawling suite of tools connected to the same common database used by sparkfun.com that includes applications for order fulfillment, production floor and supply chain management, catalog control, and content creation to name just a few things. In fact, here's an updated visualization of the scope of our code base over time:
Have a code base of your own? Throw Gource at it to get stuff like this.
The red sections (including those red bits within the green branch) are files that make sparkfun.com go. Everything else is either Sparkle or libraries used by both Sparkle and sparkfun.com. If this video looks familiar it is an updated (and higher-res) version of one we released almost a year ago.
Could we open source all of that? This is a heavy question we've been asking ourselves for some time now. Most companies might look at internally developed software of this scope and call it a competitive advantage. Such an argument might prevent them from ever open sourcing anything. SparkFun, however, is not like most companies.
We want to open up as much of Sparkle as possible... eventually. Our reasons include providing a set of flexible, modular internals to the next company that looks like we did many years ago trying to gain a foothold. Also, by fostering a community of followers we may ultimately see some contributions from developers outside of our walls. How exactly we're going to do this is not yet clear, but every journey begins with a first step. So here's what's open so far:
Blode
This is a simple yet powerful syslog-like event server written in node.js. We use this as a base for a wide variety of things from visualizations of real-time website activity to triggering subsequent events that cascade through our system. Senior Administrator Ben built this from scratch and all developers to date have had a chance to plug into it to do some really spiffy things.
SparkRegression
This is a standalone PHP class to fit six types of regression curves to a data set in order to extrapolate trends. We use it deep inside Sparkle as the base of our primary stock management application which predicts sales and usage trends to activate the supply chain where needed. I wrote this years ago when I found no such libraries for PHP existed. It helped that I still had my numerical analysis textbook from college hanging around the office.
s3fs
This is a FUSE-based file system backed by Amazon S3. While not a 100% SparkFun project our own Senior Administrator Ben has been responsible for the last twenty or so releases of this project. We use this to plug into our S3 account where much of our binary content like product images are served as if it was a local file share to facilitate access for all content creators.
lib_mysqludf_mongodb
This week we'll be migrating to using MongoDB (another open source database) to store some of our data. Mongo offers a performance boost for things that don't require the trappings of a traditional relational database like MySQL. In order to make this work seamlessly SparkFun Developer Todd put together this library to track MySQL inserts/updates/deletes to automatically store data in MongoDB. This opens up some new and exciting storage opportunities for us.
This is a great start and we're proud of what we can share. I feel very fortunate to have a team of developers passionate about open source with a whole company backing them up! Over time we'll most certainly be open sourcing more with the eventual goal of open sourcing whole modules of Sparkle for other fledgling startups to use and contribute to.
TL;DR: The future is bright for open source at SparkFun!
reddit is leaking memes again
I don't want to live on this planet anymore.
I don't always rehash memes, but when I do, I do it on the SparkFun home page. =)
I used to rehash memes...
I tried to upvote this...no lie. I could not figure out why I couldn't for a solid 5 seconds.
BTW this post is just great, congratulations to you guys, you're awesome.
I have my personal dream that someday Sparkfun is going to use a free Software CAD for their boards, like KiCAD. Someday... someday. I understand the implications of this, for your business and customers, believe me, but respectfully I think it's a double standard to create free hardware in proprietary software, especially when you expect the world to learn, reuse, reinterpret and improve the knowledge. Michael Ossman, creator of the Ubertooth One, created and awesome board with free Software, so I think the technical capabilities are there.
Sparkfun uses EAGLE which is free for viewing any design, you just have to buy in to "make" bigger higher layer count boards. That's understandable. The guys at Cadsoft gotta eat too. Additionally if you just wanna do home development a standard license is only $125. That's nothing compared to Altium or Mentor Graphics etc.
To me, open source is more a principle issue than a cost one.
I've always had a love/hate relationship with EAGLE as it "seems" open because it has a free (limited) version but it is not, while on the other hand, it is such a common software!
I totally see people's point of EAGLE being their preferred tool but often comparisons are only valid for the one making them. Kicad works for me (professionally and "philosophically") and I prefer it ... maybe if I had learned to use EAGLE first my opinion would be different, who knows.
In the meantime, I "survive" having a crippled version of EAGLE just so I can use the ULP to convert libraries to Kicad format and work "properly" ;-)
I share cjenkins' dream ... the one about Sparkfun using open source software for board documentation ;-)
Great job. Keep up the good work.
Either this was heavily edited, or all the other replies to this *(DBaron's) post are really for cjenkins' post.
It was edited - it talked about opensourcing Eagle, if my memory serves me right. The little asterisk (*) next to the timestamp indicates the user edited their post.
Indeed. While Eagle is enough for most hobbyists working on small-ish PCBs, it isn't exactly open-source and the free version has some deliberate limitations.
While there some conversion tools out there, having an official SparkFun parts library for a fully open-source package like gEDA would be really cool. And same for the various product schematics.
We're looking forward to EAGLE's upcoming migration to XML, which should open up all sorts of data-manipulation possibilities.
Yeah, I'm looking forward to that too!
Right now sending someone one footprint is difficult. I'm hoping that with XML, we can just merge a single footprint into a library no problem. I have some footprints in my personal library that people would want, and plenty they wouldn't. I'd like to be able to share some!
If I remember correctly, doesn't SFE have a GIT repo for the Sparkfun.lbr that people can branch off of and make their own footprints for ICs?
We do indeed: https://github.com/sparkfun/SparkFun-Eagle-Library
I assume you're talking about EagleCAD. The software itself is free, but limited. SFE does not produce EagleCAD at all, it is Cadsoft. If you want EagleCAD Open Sourced, talk to the folks at Cadsoft.
Yay all of the things!
These are the best kinds of open source projects anyway. One company supports it directly, ensuring it doesn't die, but since its open source other people can contribute if needed. Everyone wins!
Y U NO LIST OPENLOG?
Love it, Love it, Love it
I love open source stuff. VLC Media player in my opinion is the best media player out there and it's open source. But usually open source only works well for small and medium sized projects. Start getting into large projects and it starts to fall apart because either you have to recreate the wheel or you have to modify an existing wheel that may or may not exactly fit into what your doing, plus if you get stuck there's limited support. Open source works for hobby projects and things like that, but lets see them open source a 20" LCD Monitor or something.
I'm sure I've read that somewhere already... oh wait yes, this are the traditional arguments proprietary Software companies has been using against free software for so long... come on... do you really believe that? So.. Android, Linux, Apache, Arduino, PHP, MySQL, LibreOffice, Python, Ubuntu, etc, are "small" projects?
What I really meant by that was that I'll use open source for small projects, but once I start to build something complex every time I try and go down the open source route I get driven crazy with how shoddy a lot of it is built, how much I have to modify to get it to do what I want and how much time I have time I spend working on a sub problem instead of the real problem that I want to solve. I'd pick Windows over Unix any day. I'd also pick Visual Studio over any other open source java GUI that I've found. If I was going to mass produce something I wouldn't put an open source piece of hardware in there because it's not designed for exactly what I'm doing.
I also don't think android is really open source. Google makes money off the Android Market Place which wouldn't be operating unless it had Android so they're playing the famous give away the Razor and sell the blades market strategy. And without the heavy backing of Google Android wouldn't be anything.
Jakkjakk, I don't quite understand your point about project scale and reinventing wheels. Certainly it could be argued that much of what we're building internally, software-wise, is reinventing a wheel that's been built several times over but that can be argued regardless of whether we open source it (to say nothing of the uniqueness of our wheel that no other wheels have).
The nature of open source in the business sense for large-scale projects is one that's fairly new and still being explored. There's a lot of good info out on the nets regarding this (Fred Giasson's recent musings provide a great summary).
In general though I must respectfully disagree that open source as a concept is limited to small/mid-scale projects. Open source is a tool like any other and when applied to a project of any size can make or break it. It all depends on how it is applied that makes all the difference.
Frencil,
Where have you been these past years?
Even major companies like Microsoft and Apple have parts of their code bases rooted in open source. The first TCP/IP stack for Windows 95 was pulled right out of what is now FreeBSD. The kernel for the OSX is from FreeBSD. Every major Unix shares code base roots with FreeBSD. That is 20 to 30 years worth of some form of open source being involved in the computer industry! Then there is Google. Google owes its existence to OSS technologies. Similar story with Facebook and I am sure others. All based on OSS tech to a major degree.
cjenkins,
"Android, Linux, Apache, Arduino, PHP, MySQL, LibreOffice, Python, Ubuntu"
Yes, those are all small projects compared to FUD.EXE. FUD.EXE even says so in the the About box so it must be true. :)
What exactly is Linux? Is that some sort of *nix based text editor? ;)
Hey SF, can we get a -1 button? Linux is NOT a text editor! I'm running Ubuntu 11.10 Oneiric Ocelot and it's holding up my browser so that I can write this rebuttal. Linux is a catch-all term for open source operating systems that are based on Unix. Linus Torvalds wrote the first version and Ubuntu is a particular distribution of Linux (or "distro").
Some common distros (This is VERY incomplete, it's just from memory):
* Debian
* Knoppix (based on Debian)
* Ubuntu (based on Debian (a lot of distros are))
* Puppy Linux (can be run from RAM and based on Knoppix)
* Damn Small Linux
* Red Hat family (commercial distro xP)
* Linux Mint
Each distro often has several "flavors" (different desktop/window managers such as GNOME, KDE, xfce, etc. or versions based on another distro, or smaller, or faster, or meant to run on netbooks, etc.), but this is getting long and I'm probably putting Wikipedia out of business.
Try not to be so narrow-minded.
He was being sarcastic.
Also, you forgot CentOS and Fedora, which are the free version of RedHat.
I said that t'was off the top of my head. Hey, that rhymes! Now you owe me a dime!
Some other time.
Or you could just gimme a gold mine.
Yes, that was sarcasm, and very funny jeje
FUD.EXE: Fear, uncertainty and doubt, related to the makers of some operating system (hence the EXE). Also is the name of a malware :P
Ummm...I was thinking of some other "text editor"...
Man, this thread has been the funniest I have seen in a long time. You guys that hand around here have a good sense of humor.
"Ubuntu 11.10"? You mean "Ubuntu OSX"?
This is what I just entered into the terminal:
sudo apt-get install idioticcommentblocker
Yeah, Ocelot might look like OSX, but it's different. I used to be a Mac nerd (and still am to a certain extent), but I like being able to customize my environment more.
Linux is fuuun. Even though the distro I use might be "a wimpy cop-out for pathetic beginners". Hey, even 1337 u&3rh4xx0rs have to start somewhere!
There advantages to a Debian based system:
sudo apt-get install wontfreakinginstallsomeothercrapwithoutaskinglikeasearchbarinfirefoxwheniupdatejava
sudo apt-get install willtellmewhenitwillblowawaymysystemandmakeallmyothercrapstopworking
sudo apt-get install isgenerallyfromatrustedsourceanddoesntincludeallsortsofbadwormstrojansandviruses
I know,I know, I am enjoying this waaaay toooo much! :)
I just love *nix based systems. Because if you have a lapse of sanity you can always type: whoami
Yeah, well I tried it out and my computer got about twice as slow. Immediately started installing arch.
I wget what u did... :-P
Chill_Bill,
You made my day. :) I came up with that by looking at the word: "lin" for line editor and "nux" as implying *nix. Sorry for the confusion.
Glad to see a fellow Linux Fanatic around here.
Apology accepted.
...Captain Needa.
Wait, what?
Preaching to the choir here, Demolishun. What I mean to say by open source being newly explored territory is in reference to mainstream "traditional" businesses. The companies and organizations you rattled off get it, and despite some of them being major players that is still hardly a plurality of business entities worldwide today.
If I had a nickel every time I tried to explain our motives for open source to others in the business community who just looked at me funny and wondered where the money would come from I'd already be living in my solid gold moon fortress. The tide within our sphere turned long ago but that transition is only just beginning in wider circles.
Ah, I get it now. Yeah, people have been using this stuff for years without a clue to what has taken place over the years. Sorry I misread the intent of your post.
I was in a meeting with IT at an a previous employers and was defending the fact that I had an Ubuntu server. The IT department manager literally said, "We don't want systems based upon gaming systems on the network." All the IT people in the room that worked for this person gasped like they could not believe how ignorant their boss was. All my engineering counterparts were shaking their heads.
Edit: I forget to mention they predominately used Redhat for their servers with that IT department.
So even people who are supposed to be in "the know" sometimes don't get it either.
"We don't want systems based upon gaming systems on the network."
So... nothing from Microsoft, then? ;-)
Gaming systems?!?! OMG Ubuntu servers are not XBox 360s.
You were right to put that guy in the rear view mirror.
I think you hit on precisely why Open Source sometimes isnt feasible. Right Now Today would you release all of your intellectual property (finances, code, documentation, vendor contacts, manuals, etc, just let us have read only access to your server) to the public? Probably not. We promise in the future we will open source, but why not right now?
I love open source. My company sponsored the OHWS last year. However, in regards to breaking into a market, developing a business, ultimately there is probably some intellectual property you want to maintain so you can compete. Sparkfun is a different company, but I believe this post shows that while OS is awesome, it also can never truly be sustainable at the root of what is "open".
Would you say that the different Linux distributions are small or medium sized? Or are you talking strictly hardware here?