Month: January 2011

Categories
security server admin

ModSecurity updated anti-spam marketer rule

After a little tweaking, my rule is growing, and proving extremely effective:

# bad websites: domains which regularly or overwhelmingly feature spam
SecRule REQUEST_HEADERS:REFERER “http://[^/]*(yijiezi|yourhcg|lukejaten|squidoo|answerbag|jvlai|chaohuis|cledit|bait|lukejaten)” “t:lowercase,deny,nolog,status:500”

# porn and gambling: they make much cash out of random visitors
SecRule REQUEST_HEADERS:REFERER “http://[^/]*(holdem|poker|casino|porn|girlz|pussy|penis|babe|exposed|sex)” “t:lowercase,deny,nolog,status:500”

# fake / illegal designer clothing and luxury goods
SecRule REQUEST_HEADERS:REFERER “http://[^/]*(shop|store|cheap|gossip|handbag|money|deluxe|sunglass|chanel|replica|buy|sale|furniture)” “t:lowercase,deny,nolog,status:500”

# celebrity gossip and trying to make money out of children, I guess
SecRule REQUEST_HEADERS:REFERER “http://[^/]*(miley|bieber|pokemon)” “t:lowercase,deny,nolog,status:500”

# side-effects of Republican America?
SecRule REQUEST_HEADERS:REFERER “http://[^/]*(health|dental|pills|treatment|seller)” “t:lowercase,deny,nolog,status:500”

# side-effects of weakly-regulated investment markets?
SecRule REQUEST_HEADERS:REFERER “http://[^/]*(forex|realty|invest|loans)” “t:lowercase,deny,nolog,status:500”

# the people that created this problem
SecRule REQUEST_HEADERS:REFERER “http://[^/]*(seo)” “t:lowercase,deny,nolog,status:500”

# webhosting and bodybuilding: apparently, these industries are as commoditized as porn and gambling – LOL
SecRule REQUEST_HEADERS:REFERER “http://[^/]*(download|hosting|videos|bodybuilding|bodybuild)” “t:lowercase,deny,nolog,status:500”

Incidentally, I looked into using wordlists for this, but they don’t work. The most effective anti-spam is to look at the domain-names – these sites are trying to get good rankings for their domains, not for specific pages. Apart from the spam-friendly sites, where it’s a combination of both.

So .. sadly … we need the regexp so that we can target the domain-name specifically. If ModSecurity were better (documented) I’m sure it could easily do that. I’m suspicious it *does* do that, but with their shotgun approach to documentation, it could take days or weeks to discover it if so :).

Categories
Web 0.1

Web 0.1: Ordnance Survey / UK govt

I think it is a fantastic and wonderful thing that the complete, detailed, maps of the UK are now free for all commercial and non-commercial use. This is a long way ahead of any other country – these maps are many times more detailed and accurate than e.g. the Google Maps / Yahoo Maps / Streetmap datasets.

(PS: these days, the excellent OpenStreetMap (which works in every country – and I wanted to name-check here for anyone who isn’t aware of it already) has advanced so much that it’s seriously encroaching on the OS … why did we have to wait until the OS was heading towards obsolescence before making it free? Sigh)

(it’s just a pity it took so many years to reach this point, when e.g. in the USA, NASA has been making their content public domain for decades. All those high-res photos of space, nebulas, planets, etc – all free. For everyone)

But … it’s a pity they couldn’t find competent web-developers for their site http://www.ordnancesurvey.co.uk/:

Apart from the “I’m too lazy to write a web form properly” bug there, it also begs the question:

Why, in 2011, are you forcing people to use *EMAIL* to get a download link, instead of just downloading direct from the website?

I can think of a few possible explanations, but they all have simple solutions. So … I guess they’re all wrong. Otherwise, why hasn’t the OS done any of them :) ?

(oh, BTW: Ordnance Survey folks, you might want to run through your email-marketing database, and prune out any accounts you just created for: *you*are*incompetent*@*.com . Your crappy web-form not only failed to accept legal addresses, but it happily accepted email addresses that were blatantly fake)

And so … we have another Web 0.1 award :).

Categories
security server admin

Safe login on OS X: using an SSH key from a USB key/thumbdrive

I like computer security to be EASY and SECURE.

I hate passwords, and I use them rarely if at all. Instead, I use digital keys as much as possible (i.e. something based on a physical key stored on a removable USB drive that I take with me wherever I go). Like using a physical key, it’s much easier.

Sadly, OS X has a version of SSH that tries to be “too clever” while actually being “annoyingly unhelpful”. If you attempt to use a key from a removable drive, you get this error message:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: UNPROTECTED PRIVATE KEY FILE! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0777 for ‘login-key-for-tmachine.ssh’ are too open.
It is recommended that your private key files are NOT accessible by others.
This private key will be ignored.
bad permissions: ignore key: (key-name)
Permission denied (publickey).

(emphasis mine).

While it’s delightfully verbose, and tells you exactly what’s happened, it’s also a bit misleading. It says “WARNING” when it actually means “ERROR”, since the ssh system at this point deliberately stops itself. But, more importantly, it’s an error that you cannot evade under OS X. With OS X, all removable media has “Permissions 0777”.

Fortunately, there’s a workaround. Using this good but not-quite-detailed-enough article, I got most of the way there.

I had two problems, things that article omits.

Firstly, you are no longer “allowed” to edit /etc/fstab on OS X. Don’t try it. Instead, there’s a new command-line editor called “vifs” (hmm. vi-for-fstab, perhaps? :)) which works fine.

Secondly, the USB Drive I’m using has a space in the Label name. /etc/fstab uses spaces as a reserved character (I knew this), but … what do you write instead? (I didn’t know this).

I tried (and failed with):

  1. “My Drive”
  2. My\ Drive
  3. My Drive

…and with some creative googling, eventually found an example fstab with spaces in a label name. Aha!

  1. My\040Drive

i.e. replace spaces with “\040” (I’m guessing because it’s so low-level they’ve decided to “assume” unicode in all escape sequences)

…and now it all works as intended. Yay.

Categories
design marketing security

Identity theft, exploitation, and Gravatar

There’s a growing problem right now with Facebook Connect: it can silently log you in to websites that you *don’t want* to share your private data with. I saw a funny example last month where a porn website had integrated Facebook Connect … so when you visit the site, one miss-click and you’ll broadcast to all your work colleagues your embarassing love of HardCoreGrannies.

But there’s another example right now that may be worse, and is definitely food for thought. Facebook doesn’t broadcast your data – not to protect your privacy, but to prevent competitors getting access to data they are currently making money out of themselves. By contrast, there’s Gravatar: these guys take your private data and give it away to everyone – and they refuse to stop doing it (I’ve asked, directly, and they refused. They had no reason to refuse – they knew my identity, they knew my request was valid, and I believe under UK / Europe law it would be *illegal* for them to refuse. But … they’re American, and I guess all they care about is money).

So, for instance, I just had one of my online identities ruined by Gravatar. A website that I rarely use recently “upgraded” and implemented the gravatar system – and immediately took a private account and publically broadcast that I was the owner. They didn’t ask me, they just went ahead and did it. Like many web developers, I’m sure they had no idea what they were doing – few seem to be aware of the scam that underlies Gravatar.

Fortunately, I’m not going to lose something massively important, like my job / marriage / life (c.f. the news stories when Google Wave launched), but the website owners had no way of knowing that. They’ve just unleashed this upon their hundreds of thousands of users; what are the chances that one of them will be affected?

(incidentally, if you’re a website owner, I strongly recommend you think twice before adding Gravatar (or any of the clones) to your own site. I don’t know if anyone’s been sued for it yet, but I’m sure it’ll happen eventually)

There are two halves to the problem. Gravatar is fundamentally a violation of privacy: they take your data and give it to *everyone* without you knowing. So what? That’s the whole point of the service! Yes, the Gravatar author is a little incompetent (c.f. OpenID for how he *should* have implemented it), but otherwise there’s no problem, is there? In theory … if you voluntarily sign-up for it, it’s all OK. Isn’t it?

Well … maybe not. They won’t let you (the user / owner) control that flow of data. What happens if you change your mind – can you delete their data? Nope. Why? I’m not sure, but I would guess: If you did that, you’d undermine their ability to make $$$ out of you. You can (theoretically) set your pictures back to empty. But …

…But there’s a second half to this. I believe most people are on Gravatar because WordPress “gave” the user’s private data to Gravatar. That’s a nasty mess right there; what does WordPress’s privacy policy say? Again, when they acquired Gravatar, they apparently didn’t ask their users what they wanted, they just forced this privacy violation on them. Back then, it didn’t have much effect (Gravatar itself was relatively unknown / little used), but as Gravatar gets used more widely, the problem becomes more acute.

And here’s the rub: Gravatar’s staff refuse to adhere to privacy requests because (precising / summarising): “you have to use your wordpress.com account”. What if you don’t have one? “you must have had one in the past and we won’t help you. Go away, and stop bothering us”.

Meanwhile, WordPress refuses to send password details to anyone, ever. A wise security decision in some ways (e.g. many people use the same password on multiple sites. Doh!). Your only choice is to delete the password and recreate it.

Is that a problem? Sadly, yes. Because (due to some very short-sighted / stupid marketing decisions by the WP folks) there are lots of admin systems – e.g. anti-spam – that are run off people’s WordPress accounts. So far as I can tell, no reason exists for this *except* to harvest email addresses and try and lure people onto paid WordPress.com plans. Further, WordPress uses an archaic password-based system (instead of e.g. Yahoo’s permission-based API – which, again, is how WP should have implemented this) – so if you change your password, all those websites will break.

Summary

These services are a nice idea in theory, but when you get terrible implementations like Gravatar, combined with lazy / stupid staff, the user does pretty badly. They get screwed, they get patronised (just look at the Gravatar.com FAQ; they’ve cleaned it up in the last 12 months, it’s no longer so actively offensive as it used to be, but it’s still pretty bad), and many times they don’t even know about it until the violation is widespread.

And, ultimately, any website that uses this system is in danger of losing badly if it goes to a court-case. I’m not a lawyer, but when there are industry standards for user-controlled privacy (OpenID), and specific laws demanding that Gravatar honour the requests it currently refuses (UK Data Protection Act, for instance), I suspect a court is unlikely to look favourably on a website claiming innocence. Ignorance isn’t generally a valid legal defence.

But how much damage do these systems do to themselves? If Automattic were a little less greedy, or a little less selfish, would a lot more people embrace the idea of sharing their identity openly? Will OpenID provide a gravatar-replacement that doesn’t shaft the user, and will that take off much bigger than the original?

Personally, I look at recent events like Google Wave, and Blizzard’s “forum identity = credit-card name” – and the s***storm of angry users in both cases – and I suspect these privacy issues are much more damaging than corporates expect. Which is good news: the world appears to be slowly waking-up to the abuses inflicted upon them in the digital world, and the importance of keeping certain things (passwords, email addresses – and now, finally: identity) sacrosanct. And that is definitely a good thing…

Categories
photos

How good is Canon’s new 550D with old, cheap lenses?

Pretty good (shot without tripod, a few days ago):

Using this 10-year-old lens (70-300, no image stabilising or anything fancy). I wondered if my old (pretty cheap) lenses from my film camera would work OK with a brand new digital camera. Yes, although I get a lot of chromatic aberration at wide apertures, sadly.

So far as I can tell (and dpreview.com seems to back this up), Canon is still the best manufacturer of digital cameras for low-light situations, bar none. Which covers an awful lot of the interesting photos out there. Although … in this image … the camera kept trying to expose for “space”, so even asking it to do a full -5 stops underexposure, the moon still whited-out. In the end, I had to go to fully manual (on Canon cameras, I’ve needed to do that fewer than 5 times in my life) and force each of: Exposure (1/200), Aperture f/9.0, ISO Speed 100. This is also the first time I can remember where I’ve “had” to manually change the ISO speed on a digital camera.

Categories
security server admin

HOWTO: Prevent SEO scam Referrer traffic … AND … Install Mod-Security on Debian

UPDATE: there were several bugs in my original version – by Debian standards, ModSecurity is damn hard to configure correctly, mainly because the Debian packager has left out so much that’s essential! This version is fully tested and working…

Mod Security is an awesome, open-source product for Apache that will protect your webserver against attackers, using a custom rules-language that lets you easily filter for any kind of website attack. Even better, it comes with a pre-built (and regularly updated) set of “official” default rules for cutting out the majority of common internet attacks.

But, pretty shocking … I tried 10 different tutorials / HOWTO’s for this, and each one was wrong. Out of the 10, 6 of them lead to fundamentally insecure / misconfigured systems.

Mostly it’s the vendor’s fault for providing huge long-winded webpages in place of basic install instructions. Partly, it’s the Debian packager’s fault for both mis-packaging, and also “forgetting” to document what they’d done (e.g most of the README’s are empty. Grr!). Whatever. Here’s my HOWTO for doing it correctly, and picking up the excellent default security rules, that *should* work with most installs of Debian.

Continue reading “HOWTO: Prevent SEO scam Referrer traffic … AND … Install Mod-Security on Debian”
Categories
entrepreneurship

Get VC funding for your startup: the process

One of the most useful (and short) posts I’ve ever seen (*) on raising VC money. This post from Mark Suster encapsulates key things that every VC knows and feels is so obvious they wont even mention … But which new entrepreneurs have no way of knowing:

http://www.bothsidesofthetable.com/2011/01/11/going-to-raise-vc-heres-a-primer-on-process-people-deck/

…and if you’re raising money in europe (by which i mean “london”, in practice), i encourage you to benchmark your experience against this list.

There are still, even today, plenty of so-called VC firms in London whose processes are opaque, elongated, archaic, or pointlessly troublesome. If your VC wont stick to this process demand to know why not – and ask yourself how much trouble it will cause you down the line?

E.g. If your VC is a spinout from a London hedge fund, they may have an investment banking twist on process, that anyone from the city would recognize, but whose origins are in servicing a very different audience from entrepreneurs.

(*) – of course, im assuming you read http://venturehacks.com already. If not, youve got a lot of reading to do, and probably need to start again from scratch on your funding strategy :).

Categories
games industry games publishing

Codemasters forgets to renew codemasters.com?

Oh dear..

Categories
fixing your desktop programming

How to start Firefox fast on OS X

When you switch on your Mac…

Click on the network icon (e.g. the wifi icon for most people) and click “Turn Airport off”.

Then start Firefox.

Click on the network icon and select “Turn Airport on”.

You will find that Firefox starts up 10-100 times faster, with fewer crashes, and OS X will run faster, with fewer delays for the next couple of minutes.

Why? Firefox caching ain’t doing what it should be doing…although in some cases, I’m still not sure why

(rest snipped while I check in more detail what’s happening)

Categories
entity systems programming

Help! Which computer games use Component/Entity Systems?

For this page:

http://entity-systems.wikidot.com/start

…I want a list of published / self-published games that were built on top of an Entity System.

I know a few off the top of my head (I think – I’m going to mail some of the authors and double-check), but mostly I have no idea.

So … if you know of any, please add them to the wiki on that page.

BONUS POINTS

…if you can provide *any* of the following info:

  • Link to a public interview / post-mortem about the game that mentions the team’s experiences with ES
  • Names of any programmers and designers that definitely worked on the ES part of the game
  • *SUPER BONUS*: link to a description of how they designed/implemetned the ES part
Categories
fixing your desktop

Editing Adobe Photoshop files on OS X / linux / Windows (free)

Most professional artists don’t pay for their software (their employers do), and PSD files are the main interchange format for high-end graphics.

But PSD isn’t always possible to open or edit. Adobe’s crappy copy-protection refuses to run on some of my computers, and CS is far too damn expensive for mere mortals, so I can’t always use Photoshop to edit files.

OS X makes this a little easier – it has an excellent built-in image-and-PDF viewer (Preview) which effortlessly (and VERY fast) opens PSD files. It will even export those images to flat PNG, with a 100% success rate. But that’s no use when you’re doing complex graphics (e.g. designing GUIs for mobile-apps) and need to do layer-by-layer manipulation.

Free editors

I’ve tried many editors, both free and commercial, and I’ve found Inkscape (free) and Adobe’s Illustrator/Photoshop (expensive) to be the only ones worth mentioning. Inkscape works great on all major OS’s, too.

BUT … Inkscape still doesn’t support PSD files. This is pretty bizarre – except that Inkscape development has stalled / slowed to a crawl over the last 12 months, and I think they’re suffering the open-source problem of a temporary (long) drout in volunteers.

(interestingly, Firefox is about to release a new version of the browser that displays SVG files natively. SVG is the file format that Inkscape was “invented” for editing – so I suspect Inkscape will see a surge in interest during 2011)

NB: please don’t mention the GIMP. Even the latest version can’t handle simple PSD files – despite that being a “feature” of the app for almost a decade now, it still *doesn’t work*.

OpenOffice: world’s best image-file converter (!)

There are “conversion” programs out there, but they mostly all just use the same open-source backend (ImageMagick), which has long had problems with anything non-bitmap.

Then someone mentioned OpenOffice.

Huh? OO is a word process / excel spreadsheet / powerpoint replacement – why would I use that for a PSD?

Well … it turns out that OO has an excellent PSD importer built in – and, being OO, it happily exports to all major formats.

I tried some simple and complex PSD files, and where GIMP could open them all, but corrupted most of them … and conversion apps converted some, and for others just went blank … OpenOffice opened them all perfectly, and allowed me to save-out to the image-format of my choice.

EDIT: …this isn’t so perfect after all. OO has been collapsing the PSD layers on import for some of the files (maybe all). ARGH! But at least it’s opening files that the supposedly built-for-purpose software (like GIMP) fails on entirely.

Tis a bizarre and strange world – but at least I have a good, relatively quick, way of working with PSD files now, in those few situations where Photoshop isn’t available. And props to OpenOffice for being the one app that makes file import/export do what the *user* wants it to do, rather than propping-up anti-competitive business models and political ideals (which it often feels like the other apps are trying to do).

Categories
iphone

Sony love…

Despite this week’s PR disaster for Sony Ericsson, their new Android phone (due out soon) looks fantastic:

http://blogs.sonyericsson.com/products/2011/01/06/next-step-%e2%80%93-xperia%e2%84%a2-arc/

Personally, I am convinced that Sony Ericsson makes the best Android phones available, bar none. e.g. although many people prefer the X10 full size, I still think the Xperia X10 Mini is one of the best phones I’ve ever used (modulo some terrible software, because Sony can *never* seem to get software right).

As a consumer, I love it – it’s almost perfect (I just wish it had a high-quality camera, instead of a pre-iPhone quality camera).

As a developer, I’m frequently impressed by just how much it packs into a tiny form-factor. Every time I think “ah, crap – this isn’t going to work on the Mini”, I’m pleasantly surprised.

Apart from one thing, of course … Sony’s refusal to ever support current versions of Android OS.

Categories
dev-process iphone programming

Sony Ericsson betrays customers, claims it’s an improvement

Google keeps improving Android. Android version 2.2 is one of the most important releases ever – it speeds up the whole phone (every game, every app, runs noticeably faster), along with bugfixes and new features.

Sony Ericsson has caused much hate among consumers by shipping their flagship phones with OS 1.6, even when 2.1 was already available, and then being very, very slow to “allow” their customers to upgrade (the upgrade is essentially free, Google doesn’t charge for it).

So far, no news at all.

But this week, Sony’s official Twitter account posted this, along with possibly the world’s worst excuse from a mobile company:

There will be further system updates for the X10 handsets however there will not be any more updates to the Android platform.

…

We believe the features included in the Android 2.1 phone are on par with, and in many cases better than, a vanilla installation of 2.2 #X10

i.e. “that expensive phone you bought in the last 9 months will never run many of the new apps on the Android marketplace. We believe you should be grateful because it’s a really pretty phone, and who cares about apps / games / etc anyway? iPhone? What the hell is an iPhone?”

The sweet stench of cow-manure

Let’s put this into perspective…

When you write apps for Android, before you even write the first line of code, you have to choose the minimum OS version that your app will work with. The older the version, the fewer the features that you have access to. Also, the more bugs you’ll have to workaround (even Google has bugs :)).

So, you leave out some features, maybe. But there’s more: many of the features of newer versions are things that are invisible to the user, but which make development quicker, easier, or less error-prone.

i.e. going with an older OS means that the developer has to pay more, work harder, for less reward, producing an inferior product. It’s not very tempting!

And Google just released Android 2.3, so 2.2 isn’t even the “current” version any more, it’s the “old” version.

…net result: lots of developers will be making Android 2.2 the “minimum” version. Sony is actively “locking-out” their customers from using the phone they own to download and purchase (a percentage of) new apps. Today, that percentage is small – but it will get bigger every day, and Sony has declared they will “never” fix the problem.

2.2? Why not 2.1? Why not 2.0?

There’s lots of improvements in 2.2 that make developers lives easier or less expensive, but there’s a single “flagship” featuer that overules everything else. This one feature can easily cut 20% of the cost of development.

2.2 has improvements to the Java VM, that make *all* apps run faster. c.f. my own experiments, where my space-invaders game ran approx 20%-30% faster, just because I upgraded the OS.

That saved me possibly weeks or months of coding that I would otherwise have spent hand-tuning the code to increase performance. Performance isn’t everything, but … the wide variety of Android handsets means that even relatively simple apps may struggle to run at decent speed on some of the handsets out there.

Of course, if I don’t use *any* of the other features, I could still release this app on 2.1 … but all the people with 2.1 would be getting an inferior experience, and many would complain and rate the app down. It’s probably less damaging – and more profitable – for me to simply ban people with “old” versions of Android from using the app at all. That way, they don’t get disappointed, and won’t be able to down-rate it.

Categories
dev-process fixing your desktop iphone programming

Installing/Developing Android (especially on OS X)

UPDATE: updated August 2011, with more detailed / idiot-proof instructions – and a couple of shortcuts. NB: when you’ve done this install once, and checked the relevant bits into your Source Control, it becomes *very* fast/easy to re-install – it’s only long-winded the first time.

I thought I’d blogged this before – the install process from Google is appalling, and I wrote up detailed instructions for colleagues about a year ago. Sadly, looks like I never published the post. I just did a fresh install on OS X (January 2011) and the process *still* is riddled with flaws. I get the impression Google won’t fix it anytime soon.

NB: the biggest single problem here is: Version Control. Someone at Google appears to believe that version control is “unimportant” and that developers should “not be expected to make reproducible builds of their software”.

But … even if you’re a sloppy programmer who doesn’t do proper version control, you probably STILL don’t want to use Google’s auto-installer. It can’t cope with basic stuff like “internet connection drops for a few seconds while ADSL re-dials” – things that your web-browser deals with perfectly. e.g. as of Jan 2011, the Android auto-installer managed to crash Eclipse just because of a blip in net connection. Pathetic, really.

(not to mention the hassle of re-downloading hundreds of meg of data over and over again for every PC in your office … this document will let you download most of it once only, and install it multiple times)

I don’t have a perfect solution, but here’s a guide for how to (mostly) un-**** the installation process for Android development.

Developing for Android: Essentials

The main IDE for Android is the (mostly excellent) Eclipse – with some custom plugins built by Google/Android folks. Once it’s up and running, it’s definitely the best free solution. To make this work, you need:

  1. Eclipse IDE … *any* flavour
  2. Android plugin for Eclipse (called “ADT”)
  3. The version-independent Android SDK (called “android-sdk”)
  4. A specific OS-version of Android (e.g. current latest is v2.3)
    • NB: plus you need a “platform tools”, which you download/install in the same way, but is an extra file/feature

FYI: the first item is from Eclipse, the last 3 are all from Google. Although Google keeps trying to kick you in the teeth, you can workaround their foolishness relatively easily for the middle two items. The big problem is the last item. More on that later.

Installing Eclipse on OS X

What follows comes mostly from: http://eclipse.org

Once a year, I call-out Eclipse.org for the appalling user-experience of downloading and installing their software. They’ve had a decade to get this right, and they still get it so wrong.

As of Jan 2011: there’s been a big improvement recently – you can now find the download link quite easily. At http://eclipse.org/ there is an enormous “DOWNLOAD ECLIPSE” button in the top right of the page, in bright yellow. This is excellent.

Sadly, the download link DOES NOT DOWNLOAD ECLIPSE … instead it takes you to a screen that demands you choose the “correct” version out of 12 offered to you, with literally no help at all. The majority of users want one of only 3 versions from that screen. In fact, I suspect 99% of people that come via the front page only want one version.

Furthermore, there is *still* no download for OS X. There is *part of* a version for OS X, but no-one has done the 5-10 minutes of setup that would make it install correctly on OS X forever more. Instead, you have to manually muck about with your computer, your administrator account etc. FAIL.

1 of 4: download the “Eclipse IDE for Java Developers” file.

Unzip it. Unzip it *again*, because they decided to use archaic Unix compression instead of ZIP.

Now you have a new folder, called “eclipse”. You have to put this somewhere. There is (officially) no official location for this … good luck getting support if you put it in a “bad” place.

Most people drag/drop this folder to their Applications folder in Finder. This kind-of works, but again because the Eclipse team haven’t spent a few minutes making an OS X manifest (could have done this any time in the past few years!), the “Eclipse.app” file is broken. If you drag/drop that file to anywhere, it will stop working.

Instead, you have to manually create a shell script to run Eclipse. I’m not even going to go into it – if you want an Eclipse.app icon in the right place on OS X, Google for tutorials on how to make a working Alias specifically for this app (working around the broken one from Eclipse.org). Or … just live with the fact you have to navigate to the “eclipse” folder before you can start the application.

Download the necessary plugins for Android

What follows comes mostly from: http://developer.android.com

Immediate FAIL from Google: the download page lists *ONE* download file, out of the *FOUR* that are absolutely required in order to do Android development. There is no mention of the others :(. How am I supposed to install the software if you won’t give me the links? Your crummy auto-installer is no subsitute.

2 of 4: At least it will get you the SDK, so download that directly

… from the main download page

If you dig around the site, read through 3 sets of instructions that all tell you not to download anything, and keep bullying you into using the crappy auto-downloader instead, you’ll eventually find a file something like:

3 of 4: ADT ZIP file

(Jan 2011:) ADT-8.0.1.zip (if you google for “ADT-8” or “ADT-9” you might find future versions on the site, since Google refuses to put them in a sensible place anywhere). Alternatively, the latest version should always appear at the “magic URL”: http://dl-ssl.google.com/android/eclipse/site.xml.

(Aug 2011:) currently it’s on version 12 …

(usually, this download link is hidden in the section “Troubleshooting ADT Installation”. The Magic URL … I’ve never seen anywhere. You have to know how Eclipse works in order to deduce it. For the record, other companies (including Eclipse’s maintainers) put a webpage at the parent of that URL, to help normal users. This one just does a 404 Not Found)

Finally, you need the bit that Google *really* doesn’t want to give you: a specific version of Android OS to develop against. You need to find the URL for a “repository.xml”. In this case, you can see it flash up briefly when the auto-installer is refreshing the repository (the part that does this is called “the Android SDK and AVD Managers”). You can try that URL:

4 of 4: http://dl-ssl.google.com/android/repository/repository.xml

Google could easily have provided this URL in the docs, since it is the “official” index for downloading the other files you need, but for some reason they chose not to.

There’s 2 ways to avoid Google’s broken auto-installer (which you shouldn’t be using anyway, unless you don’t care about updating your own Android apps in future).

1. Create a local “fake” repository by manually reading an XML file and downloading each file by hand.

This used to work – in 2010, I used this successfully. In 2011, it refused to work. Something critical has changed in Eclipse so that it doesn’t recognize fake repositories – or I just couldn’t reverse-engineer the format correctly any more

2. OR: … download the files you want, and later manually copy them into the folder where Google requires them to be copied.

NB: the auto-installer often fails / timesout / dies when trying to do this. This is all it needs to do, but it’s rubbish. I’ve found that doing it by hand is usually less error-prone, tragically.

To download the files, you need the URL’s of each version of Android you want to support. Google refuses to put these in a webpage (why? Who knows?), but it’s not hard: open the XML link above – your browser should be able to view it directly:

http://dl-ssl.google.com/android/repository/repository.xml (opens fine in Firefox)

Scroll down – it’s a human-readable file – skipping the huge license. You’ll see a lot of “sdk-platform” sections – each of these is a single verison of Android, and contains one URL for you. e.g.:

<sdk:platform>
<sdk:version>1.1</sdk:version>
<sdk:api-level>2</sdk:api-level>
<sdk:revision>1</sdk:revision>
<sdk:description>Android SDK Platform 1.1_r1</sdk:description>
<sdk:desc-url>http://developer.android.com/sdk/android-1.1.html</sdk:desc-url>
<sdk:obsolete/>
<sdk:archives>
<sdk:archive os="windows" arch="any">
<sdk:size>46828615</sdk:size>
<sdk:checksum type="sha1">a4060f29ed39fc929c302836d488998c53c3002e</sdk:checksum>
<sdk:url>android-1.1_r1-windows.zip</sdk:url></sdk:archive>
<sdk:archive os="macosx" arch="any"><sdk:size>45584305</sdk:size>
<sdk:checksum type="sha1">e21dbcff45b7356657449ebb3c7e941be2bb5ebe</sdk:checksum>
<sdk:url>android-1.1_r1-macosx.zip</sdk:url>
</sdk:archive>
<sdk:archive os="linux" arch="any">
<sdk:size>45476658</sdk:size>
<sdk:checksum type="sha1">c054d25c9b4c6251fa49c2f9c54336998679d3fe</sdk:checksum>
<sdk:url>android-1.1_r1-linux.zip</sdk:url>
</sdk:archive>
</sdk:archives>
</sdk:platform>

For each one you want to download, look for the highlighted bit above. That gives you a filename – if there’s no OS X section, take the linux section instead.

You then append that filename to the URL you used to get the repository, i.e.:

“http://dl-ssl.google.com/android/repository/” + “[filename you picked above]”

I’m afraid you have to do this for each version of Android you want to build against – but for most people, that means “just the most recent few versions”.

5 of 4: another ZIP you need: “Platform tools”

You will *ALSO* need – while you’re at it – to grab an “Android SDK Platform-tools”. Google doesn’t mention this as a separate stage, because their auto-installer treats it as part of the 4-of-4 items above.

It’s another file with the URL stored in the same repository.xml from the above step.

However, it lives right at the bottom of the XML file, in a section that looks like this (there’s only one of them – no choices to make – just grab it):

<sdk:platform-tool>
<sdk:revision>06</sdk:revision>
<sdk:description>Android SDK Platform-tools, revision 6</sdk:description>
<sdk:desc-url>http://developer.android.com/sdk/</sdk:desc-url>
<sdk:archives>
<sdk:archive os="linux" arch="any">
<sdk:size>15398275</sdk:size>
<sdk:checksum type="sha1">292732c6a86971b95ca61ba8bf84cd9d6c2285c3</sdk:checksum>
<sdk:url>platform-tools_r06-linux.zip</sdk:url>
</sdk:archive>
<sdk:archive os="macosx" arch="any"><sdk:size>15127727</sdk:size>
<sdk:checksum type="sha1">86c958b461b6244e74b4e0489ea7cef9a681d882</sdk:checksum>
<sdk:url>platform-tools_r06-macosx.zip</sdk:url>
</sdk:archive>
<sdk:archive os="windows" arch="any">
<sdk:size>16589948</sdk:size>
<sdk:checksum type="sha1">14acb6271c71dce94bba879496cd66e09ae144af</sdk:checksum>
<sdk:url>platform-tools_r06-windows.zip</sdk:url>
</sdk:archive>
</sdk:archives>
</sdk:platform-tool>

Installing ADT (the “eclipse plugin for Android”)

If you downloaded this directly, as detailed above, then Google’s post-install instructions are weak. The download ZIP also has no install instructions with it. Here’s a short version:

  1. Unzip the ADT file (Google is happy with ZIP, unlike Eclipse)
  2. Create a “magic” folder for Eclipse somewhere, which is where you’ll store all your Android SDK’s, ADT plugins, etc. Create a sub-folder “local repositories”
  3. Move the ADT folder into “local repositories” (you will probably want to upgrade this later, so you need to be able to find it easily)
  4. (NB: sad design flaw of the auto-updater – it’s hardcoded that you MUST give each repository a unique “name”, even when this makes no sense to the developer)
  5. … NB: as of August 2011 … what follows is pieced together from Google’s incoherent documentation – you won’t find these steps written clearly + correctly anywhere on the developer.android.com website :(
  6. [aug 2011]: “Start Eclipse, then select Help > Install New Software….”
  7. [aug 2011]: “Click Add, in the top-right corner.”
  8. [aug 2011]: In the Add Site dialog, click Local (not “Archive”, as per Google’s instructions – Archive causes other problems later on)
  9. [aug 2011]: Browse and select the folder you unzipped/created/moved in step 3
  10. [aug 2011]: “Enter a name for the local update site (e.g., “ADT Plugin”) in the “Name” field.”
  11. [aug 2011]: “Click OK”
  12. [aug 2011]: “In the Available Software dialog, select the checkbox next to Developer Tools and click Next.”
  13. [aug 2011]: “In the next window, you’ll see a list of the tools to be downloaded. Click Next.”
  14. [aug 2011]: “Read and accept the license agreements, then click Finish. ”
  15. [aug 2011]: …Google doesn’t admit this, but some of the ADT components aren’t signed, which is a security hole. Eclipse rightly says: “Dude, are you sure you want to install unsafe software?”. You have no alternative, though – until Google re-builds the plugin to be properly signed, you just have to accept it.
  16. [aug 2011]: “When the installation completes, restart Eclipse.”

Jan 2011: Bizarrely, at this point the auto-updater hangs for a few minutes, randomly contacting internet sites for stuff that I expressly did NOT attempt to install. I’m not sure if this is a bug in the current Eclipse, that it “hi-jacks” the plugin-install process and tries to update the rest of Eclipse, or if it’s a “feature” of the Android plugin – that it secretly triggers additional installs from the internet.

Installing the SDK

In Google’s docs, there’s a whole page for installing the SDK.

It does not – at any point – tell you how to install the SDK.

Instead, that information is inside the page for installing the ADT plugin for Eclipse.

Genius!

To save you navigating their bizarre instructions, I’ll copy/paste them here. First, however, you need to create a local install directory for the SDK (they forget this step, of course). Put it somewhere that you’ll never accidentally modify or delete – if you do, Android apps will stop building in Eclipse until you manually fix everything again. Given how tortuous the main instructions are, I’ve seen smart people diagnose the problem but have massive trouble figuring out exactly how to rectify it.

Anyway, once you’ve picked a location, unzip your SDK zip file into that folder, and proceed with Google’s instructions:

1. Select Window > Preferences… to open the Preferences panel (Mac OS X: Eclipse > Preferences).
2. Select Android from the left panel.
3. For the SDK Location in the main panel, click Browse… and locate the directory that was created when you unzipped (Google words this badly – it MUST be the folder that came out of the ZIP – Eclipse will check the contents of the folder, and reject if you select its parent folder, or one of its children)

4. Click Apply, then OK.

Installing the “Android Platform versions”

This is the install part that matches the “4 of 4” download part above.

Take each ZIP file you downloaded, and unzip it.

e.g.:

  • android-3.2_r01-linux.zip

This will give you an *incorrectly named* folder, e.g:

  • android-3.2_r01/

…you *may* need to manually rename this using the “<sdk:api-level>13</sdk:api-level>” tag that was in the original repository.xml where you got the download URL from. Each downloadable ZIP file had a different – unique – api-level tag.

The new format (I’m not sure this is required, but it’s what the installer does, so I recommend doing it):

  • android-13/

Finally (!) … find the folder where you unzipped the SDK earlier. That will have files/folders very similar to the following:

  • add-ons/
  • platforms/
  • tools/
  • SDK Readme.txt

Take your (possibly renamed) folder – in my case “android-13/” – and move it into the “platforms” subfolder.

The ADT Eclipse plugin will automatically discover this when you restart.

…go through the same process for each Android Platform version you downloaded.

Shortcut: if you’ve done this before, with a previous version of Eclipse / ADT / Android … you can just take the sub-folders of “platforms/” from your previous install, and copy/paste them into the “platforms/” sub-folder of your nw SDK version. Easy.

Installing the “Android Platform Tools”

…this works in exactly the same way as the “Android Platform versions” above, except it goes in a different folder.

In this case, you need to take the zip file, unzip it, and rename the output folder to precisely:

  • platform-tools/

…then copy/paste it directly into the SDK folder. There shouldn’t be a “platform-tools/” folder to start with – this is how Eclipse/ADT knows whether or not you’ve “installed” it: it just looks for a folder with that name.

OPTIONAL: Downloading + Installing the Android SDK documentation

UPDATE: sorry, I missed this out originally.

Again, just like with the “Android Platform Tools”, you have to manually grab the URL for this from the repository.xml

The piece of XML in the repository.xml you need is:

<sdk:doc>
<sdk:api-level>13</sdk:api-level>
<sdk:codename/>
<sdk:revision>01</sdk:revision>
<sdk:description>Android SDK Docs for Android API 13, revision 1</sdk:description>
<sdk:desc-url>http://developer.android.com/sdk/</sdk:desc-url>
<sdk:archives>
<sdk:archive os="any" arch="any">
<sdk:size>103791308</sdk:size>
<sdk:checksum type="sha1">5dadb3b30e1f837716f8a5ef840e31abddf69b38</sdk:checksum>
<sdk:url>docs-3.2_r01-linux.zip</sdk:url>
</sdk:archive>
</sdk:archives>
</sdk:doc>

…then take the zip file, unzip it, and rename the output folder to precisely:

  • docs/

…then copy/paste it directly into the SDK folder.

Done!

Phew! Finally! You should now be set for Android development!

Categories
fixing your desktop

Apple Brighton: Macbook Air fixed (for free)

Less than 24 hours after giving Apple my FUBAR’d MBA, it’s back – fixed, working perfectly, and with a whole new screen too.

All credit to the local store and/or Apple’s internal repair processes. I’m not sure how much we should praise Apple overall for this, given it’s allegedly their fault in the first place (see below), but seriously – less than a day to get a laptop fixed? For free? Even though the warranty had expired? That’s awesome.

Repairing the hinge, out of warranty

NB: the reason this is such a big issue is that the hinge is integrated with the screen, so for Apple to fix this one tiny component, less than an inch long, requires replacing the most expensive part of the laptop at the same time. Apple’s estimates of the cost have been almost $1,000 for the repair, according to some reports. That’s enough to buy an entire new Air instead!

On Apple.com, there are 12 pages on this one topic, with many people who’ve reported the same issue, most of whom have had free repairs, even if their warranty had expired. Something to note: *many* of the failures happened either on very new laptops, or laptops just at the end of their warranties / recently expired. This could be to do with the failure, but I suspect it’s instead a signifier that only the people in those two situations are bothering to go online and complain.

(if your warranty just expired, and you’re afraid of having to pay for a new laptop … OR you just bought the laptop, and you’re afraid it’s an endemic issue, and you might have a chance of getting a full refund for the thing … you’re much more likely to research it. In all other cases, if your warranty is still valid, you’d probably just go to the store, get it repaired, and say nothing)

When I arrived in the store, with the obviously broken laptop, and told them what I’d read on Apple.com about the problem, I got suitably suspicious looks. Ten minutes later, after they’d looked through Apple’s internal notes, they came back and agreed to take it for repair immediately, no quibbles. Although by the sounds of it it’s *not* guaranteed – they seemed to be saying that not all the Air’s of this generation get the “free” repair.

So far as I can tell, Apple has never made an official statement on the issue. Specifically, they have NOT stated there is a “manufacturing defect” or “design fault” … although people claiming to be current or former Apple staff have (privately) described the problem that way.

…which also means we can’t be sure if the new models have been “fixed” in any way – although I’ve not (yet) heard reports of this affecting them. But given how expensive this issue appears to be for Apple, I’d bet they’ve had some level of changes made specifically around these hinges.

I wonder if this is down to legal liability – maybe there is a defect on a very few models, but there’s also a much-wider “flaw” in the design that they didn’t appreciate until the product went live (or they knew about, but didn’t have time to fix: this laptop was a major marketing piece for Apple at the time).

Categories
amusing

My new Favouritest Website Evah

Because, frankly, when the irradiated spiderfuck would anybody “desire … poor … service”?

Categories
community conferences games design

TEDxBrighton only receives positive feedback

It’s a bit mean to hilight just one culprit here – this isn’t that rare – but it’s something I’ve been meaning to talk about for ages. Sometimes, bad or broken user-interface has a direct, measureable impact on a business, due to increased customer-support costs (usually CS is paid by the minute or by the hour), or due to incorrect marketing and sales campaigns that are funded in future.

I’m not a UX person, I’m a games person. So, of course, it’s the game-design side that interests me here. Are there any free, public reports on the same phenomenon in games? I have vague memories of this coming-up at at least one of the games companies I’ve worked for, but we couldn’t find sufficient evidence at the time. IIRC, the argument was over “where is the point of diminishing returns?”, given the idea that decreased costs in support-queries justify *some* additional spending on the user-interface for a game.

Anyway, in the case I just saw, people who applied for TEDx but failed to get a ticket are auto-subscribed to a mailing list whether or not they asked for it (not unusual, but the practice always stinks of spam to me), and if they unsubscribe (manually) then their comments just get ignored: the website has been constructed so that the feedback form can’t be submitted.

I’m sure it was an accident (I’m assuming they checked the form before going live, but that it only works in one web-browser. All I know is that it didn’t work in Firefox). Either way, it would seem to ensure that “the first licensed TEDx conference” has great feedback when the licensors come to evaluate it.

Will this cost them? Not so clearly as other examples (see below for anecdotal evidence), but cost may come when they fail to take into account the negative feedback that people tried to give them, but was never received. (I’m assuming that nearly everyone who unsubscribes will have negative feedback – although in the past, when I’ve been monitoring un-sub forms, we’ve often seen 5-10% positive comments in there too. Sometimes you even see people “apologizing” for unsubscribing from your mailing lists!)

Going back to the issue of *actual* financial loss … this reminds me of a couple of talks at last year’s UX Brighton conference, and the websites listing black-hat/white-hat ways of “manipulating” the audience by making the “unsubscribe” and “refund” forms legally valid but practically impossible to complete.

In those cases, the gain/loss is usually quantifiable (allegedly). Although the practice was unanimously reviled by people at the conference, someone stood up and admitted to some experience in it – with the observation that although it “Worked” the client had then asked to un-do the process, because it increased the number of angry people phoning Customer Support (instead of using the website), and CSR staff are expensive enough that the practice had decreased profits.

Categories
community conferences education

TED: rejected

With only 250 tickets available, I guess a lot of people in Brighton will be getting one of these today:

Dear adam martin

TEDxBrighton

I’m sorry to inform you that your application to attend TEDxBrighton on 21st January has been unsuccessful.

As the first TEDxBrighton event, and offering free tickets, we have had a huge level of interest and the ticket application was very oversubscribed. … hope that in the future we might be able to offer a TEDxBrighton event with a larger capacity than the 250 this one can host.

Selection criteria in 2011…

It was an unusual process for a public event – the tickets are free, but there’s very few of them, and to be “allowed” a ticket you had to go through a review process, answering questions from the obvious, like “who are you?” to the bizarre, like “what’s your favourite web-site?”.

I remember at the time thinking it seemed very reasonable at the start, but increasingly invasive and judgemental towards the end. You want to allow/deny access based on the personal reading habits of the visitors? IMHO that comes perilously close to opening a can of worms that conference organizers should be steering clear of.

But it’s a brand with a very high reputation, so I ran with it, intrigued to see what would happen. I felt I had as good a chance as anyone – the conference is taking place in my home city, very close to where I live, and many of the TED themes have been a big part of my career and background.

Now that it’s done, I’m rather disappointed. (and of course disappointed too not to be attending the conference!) For such a high level of invasiveness, and an arrogant (although justified!) approach of “don’t call us, we’ll call you … but only if we like you enough”, I was expecting at least *some* kind of feedback :). This is the age of feedback, A/B testing, validation, and openness.

(c.f. my post the other day on UK Education and the A-Level blacklists: on the whole, those institutions that are holding-back info about public decisions tend to be frowned on these days)

What were their criteria? Who did they accept, and who did they reject? Why?

It’s not who you choose, it’s *how* you choose them

Over the years, I’ve become innately suspicious of any and all selection processes that aren’t fully “open”: with the judging criteria clearly documented in advance, and ideally with actual (theoretical) examples of good and bad submissions.

Partly … because of my own experience as a judge. I’ve judged or helped judge everything from obscure community programming contests, through game-design contests with cash prizes, to competitions giving hundreds of thousands of dollars in cash funding to new businesses.

Every time the judging criteria were given to candidates in advance, the overall quality of submissions was massively better, across the board. Every time the criteria were vague or secretive, the volume of crappy submissions was depressingly high.

…speaking of which, I still have some user-submitted game ideas from 6 months ago that I promised to review publically and critique on this blog. Every time I fire up the laptop for a long journey, I pull them out and go over them again, and I can only apologise profusely that most of them are still unpublished. A new-year resolution for me, perhaps?

Categories
games industry recruiting

A-level blacklists and the Computer Games industry

Interesting announcement from the UK schools minister, David Willetts: from 2012, UK universities will be legally required to publish their exam blacklists.

This is something I desperately hope comes into practice (apparently it’s just an “aim” right now, no telling if it’ll actually happen).

It’s especially interesting given the rich and powerful games-companies that keep banging the drum of “There’s not enough qualified IT professionals in the UK” (which, IMHO, is bullshit: they know there’s plenty, but the companies in question have a poor reputation and not enough people *want* to work for them). So far as their claims are genuine, a large contributory factor is students taking “the wrong” A-levels, and then failing to get into “the right” degree course.

For instance, when recruiting graduates for development jobs, we’d often see people with the “IT” (Information Technology) A-level, or some variant of it. Most professional programmers know that that A-level is worthless – it teaches next to nothing, and demands next to nothing. If someone included it on their list of A-levels, they were immediately downgraded in the CV/Resume pile – it begs the question “were you too lazy to do a real A-level? Why should we even consider you when there’s 2 other candidates who did a “full” 3 (or 4) A-levels – are you going to have the same attitude (work-shy) if we employ you?”.

This is not fair: many people chose that subject without realising what it signifies. I suspect the root problem is that teenagers aren’t encouraged to read the curriculum of their chosen subject BEFORE starting the A-level, and judge it for themselves. Analysis, suspicion, and appraisal (in the UK) is now banned until the age of 17: the GCSE syallabus for most subjects in the UK punishes any kind of critical thinking. Shocking, tragic, sad, but with enormous momentum of its own. In reality, changing that would be massively difficult – and anyway, there *should* be other checks and balances.

…One of which is the universities, who look at those A-levels in detail each year, and “judge” them carefully. Unlike critical appraisal of A-levels, sixth-formers considering university tend to look in great detail at anything the university has to say about entrance requirements – no cultural or educational shift is required to get their attention at this point. So, if more universities publish this info, I’m sure it’ll be seen by a great many more of the people who need it.

Incidentally, this is one of the few areas where a move to a USA-style “commercial” university system may be a big improvement over the traditional UK system. Because students would have to pay vast sums of their own money to go to uni, they’re “likely” to be a lot more critical and a lot more demanding up-front, before they spend their cash. Maybe.

Yet my own small experiences of USA undergrads suggest the opposite. There are a couple of “universities” (and I use the term very loosely) in the USA that specialise in “Computer Games Design” courses, or similar, where the students learn … nothing. They just spend the time playing games, making shitty models in Max/Maya (donated to the “university” by Autodesk’s aggressive marketing/sales team, keen to do a loss-leader and capture future users), and having the sunshine blown up their ass by “professors” with little or no qualification in the subjects at hand.

We know this for two reasons: firstly, we see the CVs/Resumes that come out of them, and they’re so bad it makes you want to cry. “Portfolios” that look like the scribbles of a 4-year-old child; self-important monologues on game-design that would make even Molyneux blush and declare “oh, how pompous!”; “code samples” of students *failing* to implement space-invaders, or tetris.

Secondly, there’s the increasing bitterness and anger of the students that have been through that system, come out the far end, and realised how much they’d been ripped-off. I won’t name names – no need: if you’re considering a college, just google it with the word “sucks” or “waste of money”, and see what happens. The guilty colleges have websites dedicated – probably even whole youtube channels – to bitching about how bad they are, from current and former students.

In those cases, even the prospect of going 10s of thousands of dollars into debt wasn’t enough to spur the students into critical appraisal before heading to uni. Which leaves me unconvinced that “paying for your degree” will solve such problems – although it will excuse the responsble authorities from taking responsibility in the future. For good reasons and bad, universities, lecturers, schools, and government are all keen to pass the buck here – and the pay-as-you-go education seems a neat way out.

So … yes; bring-on the blacklists! Share this info, info which the (arguably) morally bankrupt Examination companies would like kept buried forever (because it directly reduces their profitability). Info which successive governments had no interest in revealing (because it would draw too much attention to the severely ****ed teaching of some subjects – and lead to public demand for the government to fix something enormous it had neither the money nor the will to achieve).