Data storage is necessary for computers to work because it allows them to store and access the information they need to perform their tasks. Without data storage, computers would not be able to save any information or data, and would have to start from scratch every time they were turned on. Computers use data storage to store operating systems, applications, documents, media files, and other types of data.
When a computer is turned on, the operating system and other necessary programs are loaded into memory from the storage device. As the user works with the computer, data is continuously read from and written to the storage device. This allows the user to save documents, access media files, and run applications, among other things. Data storage is also important for data backup and recovery. By regularly backing up important data to external storage devices or the cloud, users can protect their data from loss due to hardware failure, software errors, or other types of disasters.
The evolution of computer storage has come a long way since the early days of computing. Over the years, advancements in technology have led to the development of various storage technologies with increasing capacity, speed, reliability, and versatility. There are two main categories that storage methods fall under, and if you’re a SparkFan there’s a chance you use both!
Local Storage
Local storage is storage that's well... local! To you, that is. Stored on your machine and unable to be accessed from anywhere else (for the most part). Local storage for a long time was the only type of storage, and it underwent a long evolution to become the beautiful hard drives we know and love today. Let's take a look at just a few of the many steps along the way.
Magnetic Storage
Magnetic storage was one of the earliest forms of computer storage, and it has seen significant advancements over the years. Magnetic drums and magnetic tapes were widely used in the 1950s and 1960s, but they were relatively slow and had limited capacity. In the 1950s, magnetic disks, also known as hard disk drives (HDDs), were introduced, which revolutionized computer storage. HDDs used disks coated with a magnetic material to store data as magnetized spots. They offered higher capacity, faster access times, and random access to data, making them ideal for both primary and secondary storage in computers. Over time, HDDs have continued to evolve, with improvements in storage density, speed, and reliability.
Magnetic drum memory from 1950. Source: CHM
Rack of magnetic tapes. A typical reel could hold 160 MB! Source: CHM
Internals of a 2.5-inch laptop hard disk drive
Floppy disks were another form of magnetic storage. Introduced in the 1970s, they became popular in the 1980s as a portable storage medium for personal computers. Floppy disks were small, flexible disks coated with magnetic material and were used for storing relatively small amounts of data, typically in the kilobyte (KB) to megabyte (MB) range.
These somewhat ominous floppy's come in peace (and obsolescence)
Solid-State Storage:
Solid-state storage, which uses semiconductor-based technology to store data, has become increasingly popular in recent years due to its many advantages over magnetic storage. Solid-state drives (SSDs) have no moving parts, which makes them faster, more reliable, and more durable than HDDs. (This also, unfortunately, means they don't make the fun spinny noise). SSDs use NAND-based flash memory, which allows for high-speed data access, low power consumption, and smaller form factors. SSDs are commonly used in laptops, smartphones, tablets, and other portable devices, as well as in data centers for high-performance storage.
SSDs come in all shapes and sizes
Optical Storage
Optical storage was another significant advancement in computer storage. Optical discs, such as CDs, DVDs, and Blu-ray discs, use lasers to read and write data as microscopic pits on the disc's surface. Optical storage provided high-capacity storage for multimedia content, software distribution, and archival purposes. However, with the rise of digital distribution and cloud storage, optical storage has become less prevalent in recent years.
All of history's most important work, entrusted to optical storage
The Era of the Cloud
Enter: the cloud. You may think of the cloud as a sky-dwelling aggregate of ones and zeros holding your entire camera roll and IoT data, but it is very much grounded (just not where you are). The cloud is a distributed system of interconnected computer resources that provides users with access to computing resources and data storage capabilities over a network, typically the Internet. These resources are owned, operated, and managed by third-party service providers.
Tim Cook, if you're reading this, I am NOT upgrading my iCloud storage.
Photo: Fredrik Broden
The cloud infrastructure consists of a large number of servers and storage devices that are located in data centers around the world. These data centers are designed to provide high availability, scalability, and fault tolerance, using technologies such as load balancing, and data replication.
The cloud also relies heavily on virtualization technology, which allows multiple virtual machines to run on a single physical machine, effectively sharing the hardware resources. This enables cloud providers to optimize resource utilization and provide on-demand scalability to their users.To ensure the security of data and services in the cloud, cloud providers implement a variety of security measures, including encryption, access control, and monitoring. They also comply with various regulatory standards, such as the General Data Protection Regulation and the Health Insurance Portability and Accountability Act.
Cloud Storage
Cloud storage has emerged as a popular form of data storage in recent years. It involves storing data on remote servers accessed over the internet, allowing for convenient access to data from anywhere with an internet connection.
Cloud storage is scalable, cost-effective, and offers data redundancy and backup, making it a popular choice for businesses and individuals alike. Cloud storage providers, such as Amazon Web Services (AWS), Microsoft Azure, Microsoft OneDrive, and Google Cloud, offer a wide range of storage options, including object storage, file storage, and block storage, with varying performance characteristics and pricing models.
Cloud Computing
Cloud providers offer users access to computing resources through various service models, including Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). IaaS provides users with access to virtualized computing resources, including servers, storage, and networking. PaaS provides users with a complete development and deployment environment, including a runtime platform, programming languages, and tools. SaaS provides users with access to software applications that are hosted in the cloud and accessed through a web browser or API.
With cloud computing, you can use hardware through the Internet
Cloud computing allows users to scale up or down their computing resources on demand, without having to invest in expensive hardware or infrastructure. This means users can adjust their computing resources based on their needs, without having to worry about buying and maintaining their own servers or other equipment.
Users can also access their data and applications from anywhere with an internet connection. This makes it easier for users to work remotely, collaborate with others, and access data on the go. In addition, cloud computing can be more cost-effective than traditional computing methods because users only pay for the resources they need. This eliminates the need to purchase and maintain expensive hardware, and allows users to reduce their overall IT costs.
The Cloud knows all.
Providers typically offer high levels of uptime and availability, which means users can rely on the service to be up and running when they need it. This can be especially important for businesses that require uninterrupted access to their data and applications. They also often have strong security measures in place to protect user data. This includes measures such as data encryption, network security, and access controls, which can help protect user data from unauthorized access or theft.
Challenges and Tradeoffs
Capacity
Increasing the capacity of computer storage has been a constant challenge. As technology has advanced, the demand for higher storage capacities has grown exponentially to accommodate the ever-increasing volume of data generated by various applications, from text documents to multimedia files. Meeting the growing capacity requirements while maintaining affordability and reliability has been a significant challenge for storage technology developers.
Cost
The cost of developing storage technology has been a challenge, especially in the early stages of its evolution. New storage technologies often require significant investments in research and development, manufacturing, and infrastructure, which can drive up the cost of storage devices. Balancing the cost of storage with the need for affordability and accessibility for consumers and businesses has been an ongoing challenge.
Access Speed
Access speed, or how quickly data can be read from or written to a storage device, has been a critical challenge in storage development. Faster access speeds are necessary for applications that require real-time data processing, such as databases, multimedia editing, and gaming. Improving access speeds while keeping costs reasonable has been a constant challenge for storage technology developers.
Reliability
Ensuring the reliability of storage devices has been a challenge. Data integrity and durability are crucial for storage devices, as data loss or corruption can have severe consequences. Achieving high levels of reliability, including protection against physical failures, wear and tear, and other environmental factors, has been a significant challenge in storage technology development.
Portability
Portability, or the ability to store and transport data easily across different devices or locations, has been a challenge in storage development. As users' needs have evolved to become more mobile and distributed, storage technologies have had to adapt to provide solutions that are portable, compact, and lightweight, while still maintaining sufficient storage capacity and performance.
Data Security
Data security has been a persistent challenge in storage development. Protecting data from unauthorized access, data breaches, and other security threats is crucial, especially as data privacy regulations become more stringent. Developing storage technologies with robust security features, such as encryption, access controls, and authentication, has been a challenge to ensure the confidentiality, integrity, and availability of stored data.
Technological Obsolescence
Storage technologies can become obsolete quickly due to the rapid pace of technological advancements. New storage technologies may emerge, rendering older technologies outdated, and requiring migration or conversion of data to newer formats. Managing technology obsolescence and ensuring backward compatibility has been a challenge in storage development to ensure data accessibility and longevity. Just look at the poor floppy disk :/
Pros and Cons
Choosing between cloud vs. local for your storage and computation is a choice a lot of techy people have to make every day, and the answer often depends on what exactly it is you're storing or wanting to accomplish. Here's a list of the pros and cons of each method.
Local
pros
- Fast access speeds: Data stored on local storage devices, such as hard drives or solid-state drives (SSDs), can be accessed quickly without requiring an internet connection.
- Control over data: With local storage, users have full control over their data and can manage it as they see fit, without relying on third-party providers.
- No ongoing costs: Once a local storage device is purchased, there are no ongoing costs associated with using it.
cons
- Limited storage capacity: Local storage devices have a finite capacity, and users may need to purchase additional devices as their data storage needs grow.
- Vulnerable to hardware failure: Local storage devices are vulnerable to hardware failure, and if a device fails, there is a risk of data loss unless backups have been made.
- Limited accessibility: Data stored on local devices can only be accessed from the device itself or from other devices connected to the same network.
Cloud
pros
- Scalability: Cloud storage can be easily scaled up or down as needed, without requiring the purchase of additional hardware.
- Accessibility: Cloud storage can be accessed from anywhere with an internet connection, allowing for easy collaboration and remote work.
- Automatic backups: Most cloud storage providers offer automatic backups, reducing the risk of data loss due to hardware failure.
cons
- Ongoing costs: Cloud storage providers typically charge a fee based on the amount of storage used, so ongoing costs can add up over time.
- Security concerns: Storing data in the cloud can raise security concerns, as users are entrusting their data to third-party providers.
- Dependency on internet connectivity: Cloud storage requires an internet connection, and slow or unreliable connections can lead to slower data access speeds.
The Future of Storage
Emerging Technologies
Several emerging storage technologies show promise for the future of computer storage. For example, shingled magnetic recording (SMR) is a technology that allows for higher data density on HDDs by overlapping magnetic tracks, resulting in increased storage capacity. Similarly, non-volatile memory express (NVMe) is a high-performance storage interface that provides faster data transfer speeds compared to traditional storage interfaces like SATA. Additionally, technologies such as holographic storage, DNA storage, and quantum storage are being explored for their potential to revolutionize computer storage in the future.
DNA is able to store complex information in incredibly small scales
SparkFun is in the cloud! Check out our IoT products to stock up for using cloud operations in your projects.
You can also check out our IoT related tutorials to see how you can implement cloud-based technology on your own
LoRaWAN with ProRF and The Things Network
Learn how to make a LoRaWAN node for your next long range IoT project and connect it to the internet with The Things Network!
How to Make a Magic Mirror with Raspberry Pi
Need a great project for your Raspberry Pi 4 kit? Use it to create a command center to display the weather, clock, your calendar, or even a news feed!
Monitor Sensor Data from Anywhere
Using WiFi to send sensor data from an ESP32 to a WiFi network and be able to read it from an IoT Dashboard from anywhere in the world.
Connecting Thing Plus Matter to Google Nest Hub
Connect your Thing Plus Matter board to a Google Nest Hub, as well as the Google Home app, and set up as a light device!
What type of storage do you prefer to use for your projects? What type of computer storage do you have a soft spot for that we didn't mention? Do you feel strongly about cloud vs. local?
We want to know your answers to these questions and more! Leave a comment below, or let us know on Twitter, Instagram, Facebook or LinkedIn.
Good article, though I have a few comments (based on the fact that I've been "doing" computers since shortly after Neil Armstrong took his "one small step").
First, you completely missed an historically important stage of data storage, namely punching holes in paper (either cards or paper tape). The use of punched cards for data storage dates back to the 19th century, long before any form of electronic computing, and was relatively common into the 1980s. However, it was a very bulky form of data storage, but one of the few advantages was that you could use the human eyeball to "read" the actual holes (though admittedly it took some skill to do so). I'll also mention that there were several "minor" methods of data storage that were tried, and although some such as CRT storage provided imporant "steps", their use was short-lived and easily missed.
My second comment is that there is a major, often ignored, difference between HDDs and SSDs: virtually all SSDs today are based on "floating gate" technology down at the silicon level, and any floating gate technology can only be written a finite number of times (though even some of the early floating gate technologies, in the form of EPROMs, had a life expectancy on the order of 50,000 write cycles). Although 50,000 write cycles sounds like a big number, if you have data that needs to be updated once a second, that can be hit in about ONE DAY. Magnetic data storage, such as HDDs, can be updated every MILLISECOND for YEARS on end without failure. True, the controllers for most of today's solid state storage units do try to do a form of "load balancing", moving the exact maping of changed data around so that the same cell isn't pounded on too frequently, but the limit still exists. I should also mention that there ARE solid state non-volatile storage devices which are NOT based on floating gate technology and so don't have that write-cycle limit, such as FRAMs (e.g., Adafruit's PID 1895) or MRAM (Magnetoresistive RAM), but these are typically more expensive. There are also some that "fake it", and are more appropriately called "shadow RAM" that when they detect power failure they "bulk write" from a traditional RAM to floating gate that preserves the data for the next time that power is restored.
Finally, you mention that under "pros" for "Local" that there are "No ongoing costs" -- I would beg to differ, and say that although the ongoing costs are tiny compared to the costs of buying the media, they ARE there -- included are such things as the cost of electricity to operate them and the costs of doing backups.
I'll also mention that "old-timers" like myself sometimes joke that "The Cloud" is another name for the "NSA Basement" -- suggesting that you don't know WHO has access to your data if it's "in The Cloud". (Remember, "just because you're paranoid doesn't mean that they're not out to get you!")
Great article - I enjoyed the walk down memory lane!