New SparkFun Data Service

We've launched a bit bucket for all your data.

by Nate July 10, 2014 6:00 am UTC 28

• Business
• Education
• Internet of Things
• Nate posts

Favorited Favorite 0

I'm excited! We've got a new service to offer today. It's the SparkFun data channel over at data.sparkfun.com.

Over the last 6 months I've had multiple people tell me: What SparkFun really needs is a data channel! Like a good politician I told them thanks and that I'd look into it. I never really understood why a person would need to store temperature values into a database. That all changed when I built my weather station.

After getting the hardware working I explored how to post data to Wunderground, the community driven site to "make quality weather information available to every person on this planet." Neat. But how do I push my weather data to them? Turns out there's a protocol for uploading data from your personal weather station to Wunderground.

Go ahead, try posting this link into a browser tab:

http://rtupdate.wunderground.com/weatherstation/updateweatherstation.php?ID=KCOBOULD115&PASSWORD=SparkFun&dateutc=2014-07-01+03%3A32%3A42&winddir=270&windspeedmph=7.0humidity=45.4&tempf=43.1&baromin=29.4161&realtime=1&rtfreq=10&action=updateraw

You should see a page with a single word success. You can then view the weather station data you just posted here.

I am not a software person. I can't setup up a server nor can I piece together how to spin one up in the cloud. Discovering that I could concatenate a series of strings and push live data to the Wunderground servers was a moment of complete awe. I can do this! This cloud thing! YES! Suddenly it was so easy to push regular data from my embedded system out to the world for public viewing.

But as I worked through the weather station project I wanted to monitor the battery level over time to see how the solar charger was performing. Unfortunately Wunderground doesn't accept custom fields. Once I understood that I could do an http post from an Electric Imp I suddenly wanted to be able to post data from all my projects. I floated the idea with a few other folks and the project was started.

Today we present Phant (an elephant never forgets and npm didn't have a project with that name). Now stick with me: Phant is the engine, data.sparkfun.com is our free hosting of that engine. This means you can push whatever data you want to our service. Here's an example:

https://data.sparkfun.com/input/Jxyjr7DmxwTD5dG1D1Kv?private_key=gzgnB4VazkIg7GN1g1qA&brewTemp=Too_Hot

Post this link into a browser then checkout the public feed. You did that. You! You posted data to the internet right this moment! Try changing the last bit to "Too_cold" or "56.2" or anything else. URLs can't have spaces so you might need to replace your spaces with %20 ("Happy%20Wednesday" for example).

On the Electric Imp pushing data to our service looks something like this:

part1 = "http://data.sparkfun.com/input/";
myKey = "Jxyjr7DmxwTD5dG1D1Kv";
part2 = "?private_key=";
privateKey = "gzgnB4VazkIg7GN1g1qA";
part3 = "&brewTemp=";
tempReading = thermistor.read();

bigString = part1 + myKey + part2 + privateKey + part3 + tempReading;

local request = http.get(bigString); //Push this data to SparkFun data channel

See the simplistic beauty here? All you have to do is string a bunch of sensor data together from whatever hardware you're using and throw a link out into the world. Phant never forgets them. And almost any embedded device can stick a bunch of strings and variables together!

Wait, wait, wait. I'm not buying these never-sharpen-knives. What's the catch?

Channel creation is free. We have rate limited the service to 100 updates every 15 minutes (that works out to one per 9 seconds), with a maximum data size of 50MB (we automatically throw out the older records 50kb at a time).

I don't trust you with my data.

That's cool. We feel the same way about Nest, Google, the NSA, and teeth fillings from time to time. Phant is totes open. You are free to copy and deploy your own version of phant onto any server you like so that you have complete control over your data. If you just need a place to stuff the water level on your aquarium consider data.sparkfun.com for now.

Is my data safe?

Most definitively probably. We're going to make every effort to maintain your data and provide an outstanding level of service. We'll try not to let you down but there is always a slight possibility of data loss. You can absolutely download your data at any time: CSV and JSON are currently supported.

We feel it's important to point out that SparkFun is currently privately held and we plan to keep it that way for the foreseeable future. There's nothing wrong with venture capital but being private means we don't have unlimited resource. Conversely, we don't have investors nagging us to sell or do bad things with your data. Consider the SparkFun data channel to be the neutral ground for devices to store data.

What is this phant thing?

Phant is short for elephant. Elephants are known for their remarkable recall ability, so it seemed appropriate to name a data logging project in honor of an animal that never forgets. Phant.io will give you all the information you need including documentation about creating, clearing, and deleting feeds as well as specifics about rate limits.

Why?

That's easy to answer! We needed it and we thought you could benefit from it too. All the other options we found were too convoluted to use and had strings attached. We're hoping you buy a Wifi module, a cell phone module, a RedBoard, an ethernet module or some other embedded device that captures data and needs place to put it.

For your next project consider creating a channel to log the data from your device. We sincerely hope you find it easy and powerful to use.

Checkout these tutorials for examples of how to get a widget reporting to data.sparkfun.com:

• Interface examples for Ethernet, CC3000 WiFi, Electric Imp, Raspberry Pi, and Yún
• Internet Datalogging with Arduino and XBee WiFi
• Wimp Wireless Weather Station
• Software (php, perl, ruby, shell) examples that demonstrate how to communicate with phant
• Installing Phant on a Beagle Bone Black