Even my mum uses it!

I am very happy with the choice.  In the original posts I managed to setup three home machines, one Linux one windows and a mackbook.  Since then I added the iMac that I keep at my parents.  This is a sign of how easy crashplan is to maintain.  In a nutshell once you have set it up, you just forget about it.  This has to be one of the most important things about backing up.  It does not require any maintenance whatsoever.  This is encouraging when your several hundred miles away from one of the machines.  I can use the web to see if its backed up, and get alerts if a period goes by and it fails.  I just checked it now, 100% backed up today.

What else?

The backup client allows you to set how much bandwidth and CPU its allowed to use.  I use the default as I have never found it hogging either.  I am probably lucky that so far several backups on different machines have not started at the same time.  If it did its easily curable.

In summary

If you have anything worthwhile keeping, and want off-site backup – get a crashplan account.  If you have friends with space, use crashplan for free and backup to each others machines.  In fact do both!

                          

After my terrible experience with memopal, I decided to look at crashplan.  Manung Han at Lab49 had a good experience.  Crashplan was the next best offer price when I evaluated the first time around.  This time I don’t need to go over the ADSL issues etc.  If your interested in what happened last time I attempted an offline backup read on.  I am going to keep this article short and to the point for fear of overloading syndicated websites like Lab49 with backup pages.  I will post updates to this final page as I go.

Strategy

After the last attempt I re-thought my strategy.  The new idea is to find a service that is free for the first month.  Then attempt backups with Linux, Mac and Windows.  These will not be full backups but big enough to test the system.

Crashplan feature summary

Crashplan Features

Crashplan is feature rich when compared to memopal.  For me these are the imporant ones:

• Supports Mac, Linux and Windows.
• Supports backups to Folders, Offline Disks, Other machines, Friends Machines and finally at crashplans own secure server.
• Bandwidth throttling.
• Encryption and good remote facilities.

Day 0 – Linux backup home folder 1.9Gb

Install: Very simple to install using the shell script provided.  One small gotcha.  If you do not have a version of java in the places it searches you can let it download one.  The script picked a version that would not run on my machine.  I have had the same issue myself in the past so I can not blame crashplan for this.  The solution is simple.  Remove the one provided and link to one of your own.  i.e as root or using sudo.

cd /usr/local/crashplan/
rm -Rf jre
ln -s /storage/dev/tools/java/jdk/jdk1.6.0_17/jre/ jre

After this start the backup service:

/etc/init.d/crashplan restart

Finally there should be an icon on your desktop for the gui but if not you can run this: /usr/local/crashplan/bin/CrashPlanDesktop
Once installed I had my folder backed up to crashplans server in a few hours. Job done!

Day 0 – Mac backup home folder 7.6Gb

Install: The mac install comes as a dmg file. Very easy and no problems at all. I ran a 7.6Gb backup onto the Linux machine over a 1Gbps network in less than two hours.  Then kicked off a backup to the online service at crashplan.  This will take around a day capped at 1mbps.

Day 2 – Mac backup home folder 7.6Gb

0.98 Mbps – It was throttled to 1Mbps max
The Mac backup has finished. Its now in two locations, and continues to detect changes and backup those too. Very impressed.

Day 0 – Windows backup Gb 42.2

Installed very easily. Backup started.

Day 7 – Windows backup Gb 46.4

0.66 Mbps – It was throttled to 1Mbps max and includes any pauses

152 hours later the backup is finished.  Some additional images were added during the time and crashplan picked them up too.  My verdict is Crashplan rocks.  It knocks memopal right out of the water.  The backups I have done completed smoothly with no outages.  The extra functionality is very useful and makes for an excellent product.  I will sign up once the free trial finishes.  Not only do they provide the unlimited storage, but there is quite a bit of extra functionality and refinement with the paid up version.

Excellent work, great job crashplan!

Graph of the uplink speeds during the backup.

Restoring files

The backup is finally complete several days later.  There are some final steps to go but nothing important.  For instance, I might employ crashplan to backup local offline storage, replacing my current systems.  Its time to test the restore system.

Test Steps

1. Restore files to seperate folder.
2. Compare them with the original folder.
3. The only differences should be on regularly changed files or known differences since the last backup.

This test was conducted on Linux restoring about 1.3Gb.  The restore ran for about 30 mins at around 600mbps which is reasonable for my connection over such a large distance.  There were only expected differences so the test passes.

Conculsions

I will be signing up for the online backup.  I have yet to decide if I need crashplan plus for the extra options and flexibility.  Given that its a one off cost I will probably make the purchase at some point.

                          

0.86 Gb/day 0.36 since moving to ADSL2+
I have not managed to upload anything since day 25.  The memopal service is terribly unreliable and during this period their support team has not replied to a single email.  So today I have decided to try and get them to return my money and cancel the service.  I sent them the following email.

Please could you return my money.  Your service is too unreliable for me to use.  Its been refusing to upload files for three days now.  A backup service has to be more reliable than this.

Many thanks.

It will be interesting to see if I can get my money back.  In the meantime I would not recommend memopal to anyone.

Day 29 – Promise of a refund.

Memopal have promised to return the funds I will post here when they do:

Dear Martin,

well notesd with thanks.
As promised we will provide to refund you.
Once done you will receive a notification e-mail (by the next 20 days).

with regards,
Walter

In the meantime my quest for a linux/mac/windows offline backup will continue. I have learned an important lesson. Check the service first if possible. In fact my ideal service would be one that you pay for once the 75Gb has managed to upload.

May 13th 2010

On this day Memopal finally returned my funds. I did have to chase, but in the end they paid up. Thanks Memopal.

Diary links:

Part 1

Part 2

June 23rd 2010

Received the following email and re-titled this blog entry.

Hi Martin,

nice to meet you, my name’s Lucia and I work at Memopal web marketing dep.

I am very sorry for the experience you’ve had with Memopal. We work hard to satisfy our customers and when they not like our service we always refund them. Could you please revise the “Terrible service” title of your post?

Please write to me lucia.bracci@memopal.com
I can give you a 300GB free license.

Best wishes,
Lucia Bracci

                          

1.04 Gb/day

After doing so well for a few days the memopal upload ran into a few outages again today.  Lost the service between 14:30 and 20:30 a full six hours.  This time I was around and I had no issues between me an memopal.  The problem is their end.  Once the service resumed, I started seeing a few files upload with a long period before the checks were complete and it moved on to the next file. Another rejection before it kicked in at full speed again.  You can see the issue in the Draytek router traffic log.  The yellow line is the uplink speed.  It will be interesting to see how the faster ADSL service is going to change things.  Its possible that I will still be seeing outages sometimes.

Uplink graph showing issue

Day 23 – 26Gb uploaded

1.13 Gb/day

Had a fairly decent run of uploads very stable over the past few days.  Tomorrow the ADSL will be switched over so I am turning off the backup until I have the line switched over.

Day 25 – 31.5Gb uploaded – BE excellent, Memopal needs more work

1.26 Gb/day overall.  5.5Gb/day over ADSL+2

It was fantastically easy to setup the DrayTek Vigor 2820.  This was made even easier because BE have a strong community wiki as this entry on routers shows.  I have not tried to optimise the service as yet.  BE support a variety of changes including the line profile volume in Db, and interleaving V fastpath.  I will use the default for now and get Down 8797.34 Kbps, Up 991.83 Kbps.  Not bad for an old line 1.3Km from the exchange but probably with some effort from BT to replace the line through to my faceplate things would be better.  The day before the switch I was getting this on normal ADSL: Down 5891kbps, Up 237.32kbps.  I may experiment later but not until this backup is finished.  The main thing is that the line is consistently faster and reliable.

Memopal backup reliability issues

My backup problems seem to be with memopal itself.  Every now and then the service seems to break down.  After switching to the new ADSL service and kicking off the backup at around midnight on the 24th it ran smoothly for 6 hours.  Then the service broke down again.  A few files did get through.  If I was getting a full 0.9mbps as suggested by the speed test I should be able to upload 10.7Gb in a day.  The way memopal works, a file is uploaded then there is a short pause while some processing is done followed by the next starting.  So even with no outage you would not get the full throughput.  The main thing that is slowing things down though is the outages.  Its been down 8 hours so far today and its still not working properly.  Memopal support usually get back to me by the following day and shortly afterwards things start going again.  Sometimes this happens on its own.  The fact remains though the outages are pretty awful.  I would love it if someone from memopal could comment here and give us all a full technical explanation as its still conceivable that something else is going on.  I suspect though its an issue either with the backup software or the service at the other end.  I base this on the errors I get in the log.  For instance here are a selection:

Memopal error problem with HTTPS Certificate

25/03/2010 14:08:58 – Thread=4236:     Warning: probelm with HTTPS Certificate
5/03/2010 14:09:19 – Thread=4236:     Sending error report to server: errorCode=doyouhaveit network TrustFailure errorData= – Http error (7) [Error establishing connection]

Memopal error Send segment failed with response

25/03/2010 16:09:53 – Thread=4236: Sending error report to server: errorCode=sendfile resp errorData= Send segment failed with response: resp=-2
File checksum: AD7E120C05376DA88E9E042F33EFC4F838C9644B
Segment: 39
Seg. Checksum: 1F7D9C176551EC126F3CDF23F19E4BA2ECCCD8D7
Seg. Size: 524288
Recomputed file checksum: AD7E120C05376DA88E9E042F33EFC4F838C9644B
Recomputer seg. checksum: 1F7D9C176551EC126F3CDF23F19E4BA2ECCCD8D7
File. Size: 23975003

So it seems that memopal still have some work to do even if its just explaining what is actualy causing this to its more technical customers.

Another useful metric / speed test

I found another handy speed test site www.speedtest.net.  This one has an interesting slant as they allow you to measure your speed from home to another location.  In addition a ping test is performed which does not make much difference to my backup but would effect gaming over a connection.  This means I can compare a short hop with the trip to the memopal server in Italy, Rome.  Here are the resulting tests.  As you can see there is very little difference apart from the ping test.  Interesting though.

Speed Test London - Home

Speed Test London - Rome

What next?

Well I have around 50Gb more to upload, so I will post up further pages when more information becomes available. I will also try and solicit a response from memopal as I for one am very interested in working with them to get this service working properly. Even though I have had a few issues its the way forward.

The story continues here, not a happy ending I fear!

                          

I have automated home backups onto a off-line disk. A combination of windows disk mirroring and Linux or Mac scheduled scripts keeps it both simple and automated. I just have to remember to transfer to disk every now and then for longer term snapshots. If you have got this far with your backups your doing well, but you have probably realised that without an off site backup your still vulnerable to theft and fire! Well lets hope for none of these things actually happen but if your backed up its one less worry!

Theory

There are now many companies providing business and home users with off site backup services. In theory its a great idea to use your broadband connection to upload the files somewhere safe. There are some issues though. Firstly for most of us in the UK ADSL is the only option. I have cable up to the house, but the fiber services have a terrible reputation.

ADSL is rate adaptive and asymmetric. Asymmetric means the upload speed is slower than the download. Adaptive means that speeds will be slower if your line is of bad quality or long. If you are more than a kilometer from the exchange the speed will drop off, but even if your closer poor line quality will drop the speed down. ADSL also comes in various flavors, ADSL, ADSL Max, ADSL 2+ to name a few. These all have their own characteristics and different speeds.

There are some things you can do to address line quality. Even changes at home like putting the router near to the phone link and improving the internal networking can make small differences. Ultimately you might need to get your ISP to work on BT to fix the line issues but as this shows that is not always easy.

Then on top of that, even if you have top quality and max speed, your ISP may cap your usage and shape your traffic to prevent you from effecting other users.

The problem I have to solve in creating an off-line backup is getting around 100Gb uploaded. If you have a small amount (less than 5Gb) it will be easy. Also there are free services for small amounts. The asymmetric nature of ADSL means that your backup is mainly restricted by up-link speeds. So lets look at some typical speeds:

So for 36 days probably more I will be backing up, and that also means my normal internet will be slow. This is because without spare up link capacity requests will be slow even if the download stream is not used.

As I write this I have a 0.26mbps up-link so the best estimate is 36 days to get the initial backup uploaded. But what about subsequent backups. Choose a company that gives you a system that only uploads changed files and is clever enough to only store the changes. This makes it workable because even on ordinary ADSL your subsequent backups are manageable.  Some service providers make it possible to seed the initial backup using disks and post.  This costs more and I am not sure how long the data would take to arrive in Italy even then!

What about the restore? Well this is less of a problem. For starters download is much faster (I hope your on an uncapped deal!) Some companies also offer a service to send you the data on a disk. I am sure I would pay for this if I had my hardware stolen.

What about switching to another supplier? Some suppliers now make it possible to switch without a huge upload again. They will take the initial data direct from your current supplier. This happens at much faster speeds than you can achieve from home.

Cost

This is by no means an extensive review of the available services. Prices change all the time. Many home services only support windows, so that rules out quite a few. Some are just storage on the cloud, and don’t provide a backup tool however simple. Others will be fine for a measly 2Gb but I have 100Gb to store and that gets costly very quickly. In the end at the time of writing only one company provided the combination I wanted: www.memopal.com. I bought a 5 year plan for £130.

This buys me 200Gb which is more than enough, I can purchase more if I require. The file system they use makes sure the data is on more than one disk. In addition they store versions. So if I overwrite something I can get back the original. I use svn source control for documents but images are stored in the raw state so its a good idea for those. In addition they have this clever feature that if your file matches one uploaded by another user, you are linked to that, meaning it does not count towards your storage. This is handy if you upload executable files as its possible someone else already has that file somewhere.

After calculating the above I decided I needed a new ISP. After much searching I decided on BE.  Funnily enough that actually saved me money on the ADSL service and gave me ADSL 2+ with 1.3mbps up link speed.

I needed a new router though, and wanted an excellent one, so opted not to use the one provided by BE. The BE one is pretty good actually but its just not anywhere near as good as a DrayTek 2820. This set me back £147.  A bit of a luxury perhaps but it does support cable modems should I go in that direction later.

Whilst the backup is driving this it also hugely improves my ADSL and home networking so I was happy to upgrade.

Diary – the first 16 days, 8.8Gb uploaded

0.55Gb/day

I started this with my existing service on standard ADSL. Most of my data is on a windows XP machine so I will start with that. The theory said it would be slow but in addition to that my ADSL was unreliable. After signing up and downloading the memopal backup application things went well for a couple of days. Then I started to get blocks of many hours where the application would not upload files.

Contacted memopal support on day 12 sending them log output from the backup application.

To improve reliability I re-networked at home. Bought a DrayTek 2820. A non WiFi one, I have an access point for that. Plugged it in near the phone socket and experimented with different ways of connecting using the speedtester at thinkbroadband. Oddly connecting the Wan link via a surge protector was both more reliable and faster. Go figure! Noise on the line possibly?

My line is faster and more reliable (less faults) but I am still getting problems. I have managed to upload all the small files now. The big ones all fail after uploading. There is a file CRC check and that keeps failing. This indicates the file arrives at the other end corrupt. Or perhaps there is a bug in the backup software or service at memopal.

Memopal support got back to me and requested I run some tests using some web links. I sent them the resulting numbers. Directly after this the backup started working again and the following day, they confirmed that they had sorted out the issue. Not sure what that issue was though. Thanks to memopal for an excellent response.

Ordered an ADSL 2+, 1.3Mbps service from BE. Pipex are just not cutting it this service has better support, better speed and its cheaper to the tune of £3 per month.  Its going to take awhile to recoup the other costs mind. During the next few days they sent over kit and instructions. I got messages delivered to my mobile at each stage. My new line will be installed in 7 days time. Very impressed.

Day 20 – 16.9Gb uploaded

0.85Gb/day

At this stage with maximum speed and no outages I would be up to 56Gb.  The reality is far short of that.  Only very occasional outages now. Usually just resulting in 10 to 15 minute loss in service in a few days. This could be anywhere along the link between the UK and Italy or the service at memopal. I was never around to investigate. Encouragingly the backup software just deals with it and carries on when the service resumes. Looking forward to being able to shut down the machine, the fans are driving me nuts. I have two machines in the room and the Windows one is noisy.

Day 21 – 19.6Gb uploaded

0.93Gb/day

The upload is still progressing well.  The next milestone is likly to be when I switch from Pipex to BE, 0.62mbps to 1.3mbps in a few days time.

Continue to Part 2

                          

The thing that interests me is peoples response to innovation.  It takes time for people to work out how to use new systems like this.  The more flexible it is the longer it takes people to blend a culture around it.  Wave does remind me somewhat of having my first phone that could send a text, and only having one other friend with a similar phone and the will to use it.  Lets hope google get the bugs and interface ironed out soon though as I think its going to be a fun way to colaborate.