Setting Up Your Own NAS

Network-Attached Storage (NAS) serves as a centralized data repository that can be accessed by various devices within the same network. To illustrate, think of it as a Google Drive or Dropbox where information is readily available across multiple devices. Given its accessibility, NAS proves to be an excellent solution for sharing data in either relational or NoSQL databases.

In this guide, we will delve into how to assemble and configure your very own NAS using a Raspberry Pi. Our database server will utilize a Postgres database running within a Docker container, which enhances both portability and reliability. Additionally, we will augment Postgres with PostGIS for handling geospatial data and TimescaleDB to manage time-series data. This setup will adequately address most of our daily needs regarding centralized database management.

Personal Motivation

For a significant period, I stored my data as pickle files, flat files, and occasionally in local databases. While this approach is relatively straightforward at the outset, it quickly spiraled out of control, leading to three major issues:

1. Duplicated data consuming additional storage.
2. Lack of version control, resulting in inconsistencies and mismatches.
3. Projects referencing the same data, complicating access.

Moreover, data was accessible only on my desktop, making it cumbersome to work across devices. This often forced me to commit data to Git, which is not advisable for personal projects. These complications led to considerable frustration, especially when I inadvertently deleted a project folder that was referenced by multiple projects. If only I had a NAS, I could have avoided these headaches.

Future-Proofing

Consider starting to collect various types of data—financial, weather, calendar, or from smart home sensors like humidity or temperature sensors. Storing this information in flat files simply does not scale well, hindering efficient data manipulation.

However, one might wonder: Why not just use a database hosted on my PC? While that is a valid option if you don't mind keeping your PC operational around the clock, a NAS can save you on electricity costs and reduce the maintenance associated with a constantly running PC.

Exploring Options

Packaged NAS solutions, like Synology, can be quite costly. However, they come equipped with robust specifications and software, allowing for a quick setup—just plug in the hard drive and power it on! If you're interested in such products, you can explore options here. But where's the fun in that?

Preparing the Operating System

To begin, insert the SD card into your PC. Download the Raspberry Pi Imager to flash your SD card.

1. Select Other General Purpose OS > Ubuntu > Ubuntu Server 20.10 64 Bit.
2. Choose your SD card and click WRITE.

Once completed, create an empty file named ssh in the root directory of the SD card to enable SSH access. Now, assemble the hardware by inserting the micro SD card into the Raspberry Pi and connecting the HDD, keyboard, monitor, and power cable.

Connecting to Your Pi

Upon booting, enter the default password ubuntu and change it when prompted. Execute ip a to find your IP address under eth0, which should start with 192.168... Use this IP to SSH into your Pi:

ssh [email protected]

If successful, you’ll see a welcome message indicating that Ubuntu is running. For static IP assignment, access your router’s settings and configure a static IP for your Pi, ensuring it retains the same address after reboots.

Setting Up Your Pi

With SSH access established, you can disconnect the monitor and keyboard. Begin by updating your Pi:

sudo apt update && sudo apt full-upgrade

sudo usermod -aG sudo ubuntu

sudo whoami # This should print 'root'

Next, check the file system of your external HDD with:

sudo lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT

If the filesystem is NTFS, you will need to format it to ext4 for compatibility with Postgres:

sudo mkfs.ext4 /dev/sda1 # This will erase all data on the HDD

After formatting, mount the HDD by creating a target folder:

sudo mkdir /mnt/postgres

ls /mnt/postgres # Check if it mounted correctly

Change permissions to allow writing:

sudo chown -R $USER:$USER ~/postgres

Docker Setup

To ensure portability and ease of debugging, we will set up our database using Docker:

sudo sh get-docker.sh

sudo usermod -aG docker ubuntu

sudo reboot now

Reconnect via SSH and verify Docker installation:

docker run hello-world

This command should indicate a successful installation.

Configuring Postgres with PostGIS & TimescaleDB

We will now set up Postgres along with PostGIS and TimescaleDB. Thanks to Eddie, there’s a Docker image tailored for ARM chips like our Pi. Start by creating a configuration file:

cd /mnt/postgres

mkdir data

vim .env

In the vim editor, type the following:

POSTGRES_PASSWORD=password

Exit vim by pressing Esc, then Shift + Z + Z. Kick off the container:

docker run --name pi-postgres -p 5432:5432 --env-file .env -v data:/var/lib/postgresql/data dekiesel/timescaledb-postgis:1.7.4-pg12

If you see a message indicating that the database system is ready to accept connections, you’re almost there!

Using Tmux for Session Management

To maintain long-running sessions, we will use Tmux, which allows us to detach and reattach terminal sessions. Start Tmux:

tmux

Then run the Docker container again:

docker stop pi-postgres

docker rm pi-postgres

docker run --name pi-postgres -p 5432:5432 --env-file .env -v data:/var/lib/postgresql/data dekiesel/timescaledb-postgis:1.7.4-pg12

To detach from Tmux, use tmux detach. You can reattach with tmux attach.

Congratulations! You have successfully set up a NAS Postgres Server on your Raspberry Pi, complete with PostGIS and TimescaleDB. This configuration should address most of your needs, but feel free to experiment with additional components.

Additional Resources

If you're interested in learning more about Python, Data Science, or Machine Learning, consider exploring these posts:

• 3 Common Problems with Neural Network Initialization
• Google's RFA: Approximating Softmax Attention Mechanism in Transformers
• 7 Easy Ways for Improving Your Data Science Workflow

If you’re keen on applying machine learning to investment, check out these articles:

• Genetic Algorithm for Trading Strategy Optimization in Python
• Genetic Algorithm — Stop Overfitting Trading Strategies
• ANN Recommendation System for Stock Selection

If you found this guide helpful, purchasing components through the affiliate links provided would be greatly appreciated. However, feel free to search on Amazon if you prefer.

The first video: "How to Build a Raspberry Pi NAS (It's AWESOME!!)" showcases an engaging approach to creating a Raspberry Pi NAS setup.

The second video: "Build A Raspberry Pi NAS For $35 Using All New Parts" offers a budget-friendly option for establishing your own NAS.