How to back up Dropbox automatically – daily, weekly, monthly snapshots
Posted by aaron in Linux, Troubleshooting on July 9th, 2010
Dropbox is great. Most of the time. It’s great when I need to easily share files with people while working on a project together. It keeps files in sync between multiple computers, and is the only solution I’ve found so far which works seamlessly and instantaneously. If you don’t already use it, I highly recommend it.
Dropbox - Online file backup and sharing
I’ve recently been having problems with Dropbox ever since joining a shared folder which overflowed my quota. Completely unrelated files mysteriously disappeared. Luckily I had a snapshot from before I had joined the folder and was able to restore the files.
This prompted me to set up a proper backup system which makes snapshots daily, weekly and monthly.
This guide assumes you have a Linux server somewhere with no GUI.
First, follow the Dropbox wiki instructions for installing Dropbox on Linux. The process is relatively painless, and seems to just work. At the end, you’ll end up with a “Dropbox” folder in your home folder. This folder will always have the latest version of your files.
The next step is to take snapshots of that folder at regular intervals. I chose to keep 7 days of daily snapshots, 4 snapshots taken weekly, and 12 taken monthly. In theory, this should give me access to any files I’ve accidentally deleted within the past year.
This script will run rsync to clone the Dropbox folder into snapshot folders which will be overwritten over time, so you will only end up with 7+4+12 copies of your Dropbox folder. Save this code to a file called backup.sh or whatever you want, and make it executable (chmod 755 backup.sh).
backup.sh:
#!/bin/bash # Makes daily snapshots in folders like "daily-4-Thu", "daily-5-Fri" if [[ "$1" == "daily" ]] then path=daily-`date +%u-%a` fi # Makes weekly snapshots in folders like "weekly-1" where 1 is the day of the month if [[ "$1" == "weekly" ]] then path=weekly-`date +%d` fi # Makes monthly snapshots in folders like "monthly-04-Apr" if [[ "$1" == "monthly" ]] then path=monthly-`date +%m-%b` fi # Run with "go" as the second CLI parameter to actually run the rsync command, otherwise prints the command that would have been run (useful for testing) if [[ "$2" == "go" ]] then rsync -avz --delete /home/aaron/Dropbox /home/aaron/aaron/Dropbox-Backup/$path else echo rsync -avz --delete /home/aaron/Dropbox /home/aaron/Dropbox-Backup/$path fi
Add these lines to your crontab to run the backup script at the appropriate intervals. This will run the daily script every night at 1am, the weekly script on the 4th, 12th, 20th and 28th days of the month at 1:30am, and the monthly script on the 1st of the month at 2am.
crontab:
0 1 * * * /home/aaron/backup.sh daily go 30 1 4,12,20,28 * * /home/aaron/backup.sh weekly go 0 2 1 * * /home/aaron/backup.sh monthly go
Hopefully this saves you some headaches down the road when Dropbox decides to play tricks on you.
Notes on my preferred IDE: Aptana Studio
Posted by aaron in Website Development on May 13th, 2010
I haven’t used Dreamweaver since 2002 and don’t regret it for a minute. On the other hand, I’ve been struggling to find a good IDE for PHP/HTML/Javascript development for a very long time.
Komodo on OSX is pretty good, and allows for editing files directly from an SFTP server. That combined with command-line Subversion was my preferred environment for a long time.
In the last year or two I’ve started using Aptana, on both Windows and OSX. I’ve been extremely happy with it for the most part. I’ve always installed the standalone version and it’s worked without any trouble immediately after installing. As great as Aptana is, it definitely has its quirks. One of which is the obscure way to get “upload on save” to work.
The trick with Aptana is not to download version 2! They took out a lot of the great features for some reason. Use version 1.5.1, which is nearly impossible to find a link to! Save this setup file locally in case you need to reinstall it, and bookmark this link directly to the 1.5.1 installer!
The Subversion integration is fantastic, and another really useful feature is the two-way sync in case you edit files both locally and remotely and need to sync changes between the two sets of files. If you add an SFTP site to Aptana, you will be able to edit files directly on a server without needing the “upload on save” script.
After first installing Aptana, you’ll want to install the PHP plugin. Go to Help -> Install New Software and choose the PHP Update Site under the “work with” menu.
After that you’re pretty much ready to go. Check out a project from Subversion by clicking New -> Other -> Checkout projects from SVN.
Mapping wifi access points in Portland
I’ve been logging GPS data for about a year and a half. I use an app on my Windows Mobile phone to log the GPS data, and my phone has a wifi antenna that stays in the “off” position most of the time. I met with @donpdonp the other day and he introduced me to the geomena.org project. I realized it should be possible to log access points along with my existing GPS logging. It was easy to install Airomap on my phone, so now I can log wifi points too!
So far I’ve only made about a dozen trips with the wifi antenna on, but I’ve already logged 2200 unique mac addresses, and 427 open access points. And this is only on my normal route to and from work, I haven’t made any special wardriving trips yet. Here is a heatmap visualization of the access points I’ve logged so far.
This uses a custom tileserver I wrote to provide an additional data layer to the map. Hotter spots on the map correspond to more open access points in that area.
This is a version which shows each access point as a marker on the map so you can click on them.
Wifi Access Points in Portland. Red: encrypted, Blue: open
Most of the access points appear to be in the middle of the street, because most have been seen only one time by my logger, which was in the middle of the street. As the access points are seen repeatedly from other positions, the points will adjust to a more accurate position. You can already see this happening for a few points which appear in the middle of blocks or on streets through which I did not drive.
Again, these maps are still in their infancy, since I’ve only logged a few days worth of points on very narrow routes. But it’s amazing that I’ve already driven by 2200 access points just in the normal course of the day. I’m looking forward to continuing logging data and eventually importing it into the geomena database. It’s easy to see how quickly we could map out the entire city with just a couple people running loggers!
An exciting month for Google
Posted by aaron in Uncategorized on December 8th, 2009
A recap of all the things Google has announced in this first week of December. The big ones are Google Goggles, new versions of Chrome, and real time searching, but they’ve also launched some less flashy things like a dictionary and public DNS service.
Augmented reality becomes a ... reality
Google Chrome for Mac and Linux
Vevo, a venture between major music labels and Youtube.
Liquid Galaxy: 8 screens of Google Earth
Liquid Galaxy, because eight screens are better than one.
Google Public DNS, at 8.8.8.8 and 8.8.4.4
So this is New York
The Apartment
I flew in to New York from Pittsburgh in the middle of the day. The public transportation system was relatively easy to figure out thanks to Google Maps, so I got on the train to Manhattan! I found my way to Al’s apartment a few blocks from Columbus Circle.
I stayed here for a week
Rent there is insanely expensive. It is a one-bedroom with a kitchen that was basically stuck along the wall to the living room. There isn’t even room for a full fridge, just a mini-fridge under the counter. Note to self: wait until you are rich before moving to NY.
The Food
I had some good Thai food at Room Service. I checked out Risotteria Restaurant on Josh’s recommendation. They had some pretty tasty gluten-free italian food, and breadsticks that actually tasted like breadsticks.
Cafe Gitane is a cute little French cafe in the NoLita neighborhood. I had a tasty couscous thing with hummus on top. It came in a cute little stack. This was also the first real cup of coffee I’d had in Manhattan!
The Coffee
I was very surprised at the lack of coffee shops and good coffee in Manhattan. I guess coming from the Pacific Northwest, I just assumed coffee was as popular everywhere. Mostly people were drinking “coffee in a Domo cup“, 7-11’s campaign which launched apparently right as I landed in NY. I was able to find two other places that somewhat resembled west-coast coffee shops, Gizzi’s Coffee, and Pecan. There was definitely a lack of good coffee. Or maybe an abundance of bad coffee.
The Bagels
Since I was in New York, I had to find an authentic New York bagel even though I barely eat gluten. I turned to yelp.com to search for the best place to get a bagel. I knew I would have somewhat of a reaction to the massive amount of wheat in the bagel, so I only wanted to do this once. One shot to find the perfect bagel in New York. Absolute Bagels had some pretty great reviews, so I went to check it out. It was not close, it was all the way at the other end of Central Park.
It turned out to be a little Chinese place. I was super skeptical as I walked in, but I had come this far, couldn’t turn back now. I asked them if I could have a bagel fresh out of the oven, not one from the display case. The guy tore one right off the hot stack and handed it to me. And yea, it was delicious.
Wall Street
Wall street was impressive. I don’t really know what I was expecting, but it was nothing like any idea I may have had about it. There were just so many buildings so close together, and everywhere I turned I’d see a big name like “New York Stock Exchange”, “Chase”, “Federal Reserve Bank”, etc. It also never occurred to me that the reason it’s called “Wall Street” is because it’s the street named “Wall.”
I still have no idea what those red glowing things are for. I can’t tell if they’re just decorative or if they have some sort of function.
The Subway
The Subway system, while very old-looking, is pretty great. I was always able to quickly figure out how to get from point A to point B, and the trains run frequently enough that I never had to wait long for one. The subways run to every part of Manhattan (and off Manhattan as well), and it never takes more than one or two to get anywhere. Once you’re underground, you don’t have to pay or swipe your pass again until you pop up somewhere.
I think this means they don’t want me crawling under the gate?
Unfortunately but not surprisingly, my GPS was not able to get a fix while underground. My GPS data looks pretty spotty for Manhattan, with little squiggles popping up just at the points where I went above ground.
I was able to walk around a bunch of neighborhoods, including East Harlem, Morningside Heights, Hell’s Kitchen, Little Italy, the Financial District, the Lower East Side, and I even made it to Brooklyn and checked out Williamsburg.
Flirting with Pittsburgh and the G20
Sidewalk Closed
I took the bus from the Pittsburgh airport to downtown, then caught a bus from downtown to Silas’ place. Of course since it was the second day of the G20, they had a big chunk of downtown completely blocked off with roadblocks and police. I’ve never seen so many different kinds of security in one place. City police, military, riot squad, private security, roadblocks, barbed-wire fences.
Private security force
Police Rescue Vehicle - I have no idea what the thing on top is for
It's not like there was actually anything going on
On our evening tour of the city, we drove by the University of Pittsburgh, apparently while the members of the G20 were getting a tour of campus as well. There was a ridiculous amount of security present, and helicopters flying overhead with giant spotlights shining down. We walked around a bit, staying well away from any groups of people so as not to be mistaken for a protester. You can find videos on Youtube of why we did not want to get involved in the protests.
Police roadblock at University of Pittsburgh
Here are the GPS logs from my brief tour of Pittsburgh! Silas had rented a Zipcar, so we were able to cover a lot of ground.
Tour of Pittsburgh
On a lighter note, I found a great Italian restaurant in the neighborhood. The cook was standing outside while I walked up, so I asked if he could make something vegetarian with polenta. A couple minutes later he had whipped up a tasty spinach and polenta dish.
Spinach Polenta
Failblog: Starbucks and Logan Airport
This one I just thought was mildly amusing. The screens that are supposed to tell you when the next flights are coming in are apparently Windows computers running Internet Explorer.
Internet Explorer error in overhead display
At the Starbucks in the Logan airport just outside the JetBlue terminal, I noticed this writing on the deli case:
Just because it has no corn syrup does not make it not unhealthy!
In case you can’t read the letters, it says:
- no artificial flavors
- no artificial trans fat
- no artificial dyes
- no high-fructose corn syrup
Also note the handwriting font, as if this somehow adds to the legitimacy of the statements.
I hope they don’t mind me running my GPS on the plane
Since I’ve already been logging GPS data for quite some time, I thought it’d be interesting to log as much data during flights as possible. Of the three flights so far, I’ve been able to log data on two of them. I think it has to do with the fact I had a window seat on both flights. I’m pretty sure the plane is shielded in a way that blocks the satellite signals. I’d be interested if anybody can fill me in on why this might be.
I put my phone into “airplane mode” so that it doesn’t broadcast anything, but turn on the GPS logger. I was able to keep a lock for the entire flight!
GPS at 30000 feet!
San Francisco to Long Beach
Takeoff from San Francisco
This is the beginning of the flight from San Francisco, loaded into Google Earth. It also includes taking the BART to the airport. The actual flight starts at the gap at the bottom.
I also have altitude data at every point, but I can’t seem to get Google Earth to display it. If you have any ideas please let me know!
Adventures in San Francisco
San Francisco’s BART and MUNI couldn’t be any easier to use! I had no trouble getting from the airport into the city. Once in the city, it was easy to get around as well.
Burmese Salad
On Tuesday night I met up with my grandma and uncle and had dinner at a Burmese restaurant. Their Burmese salad was quite good and quite strange as well.
Afterwards we went to the swanky Cliff House for a drink, and for the view. Unfortunately by the time we got there it was dark. and foggy. If you look really closely you can see a wave.
View from the Cliff House at night in the fog
On Wednesday night after the conference I walked around by the Fisherman’s Wharf a bit. I decided to look for any web or tech-related events happening but figured it would be a long shot to find something starting that night. A quick Google search later I found the Microformats meetup happening in less than an hour at 21st Amendment. Knowing nothing about who would be there or what to expect, I got on the next bus that direction, and hung out right outside the door.
Like any good stalker should, I spent a few minutes researching microformats and the people involved. I was able to find a couple names and photos of people who might be there, so I kept my eye out for someone to approach who resembled the tiny photos I found on my iPod. At 8:00 sharp, Tantek walked up and started poking around on his phone, which was a good clue. After a brief introduction I found myself having dinner with some great people!
Ariel and Tantek
Thanks for the great evening, guys! If I’m ever back in San Francisco I’ll look you up again!
Made it back to the house after a subway ride through the fog.
Did I mention it was foggy?
First stop: Long Beach airport. Did I step into a 70s movie?
The Long Beach airport wasn’t quite what I expected. Walking through the airport I feel like I’m in the middle of a 70’s movie. I got off the plane into the terminal which basically just had a bunch of “leave now” signs directing me out of the airport onto the street. The font used for the airline signs in the airport is old-school! 
Using Google maps on my phone, I located a coffee shop to hang out in for a few hours between flights and went to the bus stop. I was surprised when I saw the bus come 20 minutes early, so I got on. As soon as it left the airport loop I realized I was going the opposite direction. I took the bus down to Circle Center and saw a Chase branch.
I figured this was a good a time as any to break my $50 into small bills. I had to walk through a metal detector carrying my laptop bag and backpack full of metal to get inside. The tellers were arranged in a circle so that all the customers were on the outside. Is this a more effective layout for preventing a robbery?
How did I get here?
A couple blocks away was a coffee shop so I figured I’d crash there for a while and figure out how to get back to the airport.