I haven’t written anything in a while, so I thought I might begin by
sharing some ideas on how we collect test site data at Neurobat
without using FTP.
We have a Dropbox Teams account,
which gives us about north of 1 TB of storage. The idea is to install
Dropbox on each netbook that monitors a test site, and to have it save
its monitored data to its Dropbox folder. Then when/if that netbook
has internet access, it will automatically upload the data to a shared
folder on our Dropbox account, from which it will be re-distributed to
all interested parties:
X
/ \
/ \
/ \
/+—–+\
| | +———+
| |…|netbook-1|….
| | +———+ .
+—–+ .
site 1 .
.
.
.
X .
/ \ .
/ \ +—————————————–+
/ \ | ____ _ |
/+—–+\ | | _ \ _ __ ___ _ __ | |__ _____ __ |
| | +———+ | | | | | ‘__/ _ \| ‘_ \| ‘_ \ / _ \ \/ / |
| |…|netbook-2|..| | |_| | | | (_) | |_) | |_) | (_) > < |.
| | +---------+ | |____/|_| \___/| .__/|_.__/ \___/_/\_\ |
+-----+ | |_| |
site 2 +-----------------------------------------+
.
.
.
X .
/ \ .
/ \ .
/ \ .
/+-----+\ .
| | +---------+ .
| |...|netbook-3|....
| | +---------+
+-----+
site 3
For simplicity (and for licencing costs) the same user account is
linked to each netbook. For historic reasons, that account is
`nagios@neurobat.net` (yes, a Nagios server is going to monitor that
data too).
Since we don't want data from, say, site 3 to be copied over to the
local drive of netbook-1, we enable the "Selective Sync" feature on
the netbooks so that each netbook sees only a folder dedicated to that
site. The folder layout thus looks like this:
Monitoring/
├── site 1
│ └── sensors
├── site 2
│ └── sensors
└── site 3
The `Monitoring` folder is the globally shared folder, shared by
`nagios@neurobat.net` with all Neurobat users interested in analyzing
the data.
The only downside we have identified with this approach is that
whenever a new site is added, that site's data will by default also be
synchronized with all existing sites. Selective sync will only let you
exclude folders that already exist; as far as I know, you cannot ask
Dropbox to *not* sync new folders in that shared folder. And I am
apparently not the only one to find this annoying.
An alternative might have been to have a separate shared folder for
each site. However, for each new site one would then have to share
that folder with all Neurobat users. Furthermore, since all sites use
the same Dropbox user, you would still end up with that site’s data
being sync’ed all over the place.
That being said, we have found the above scheme to work out very well
in practice. The automatic synchronization of Dropbox makes it a
superior alternative to, say, hand-written FTP-calling scripts we
might have used in the past century. And if you have notifications
turned on for your Dropbox account (I personally don’t) it will be
very reassuring to regularly see your test site data being updated on
your local machine without any human intervention.