After extensive testing, they’ve concluded that the best way to work around this is to pretend to be Version 9.80. From now on, all versions of Opera will identify themselves as “Opera/9.80″ with the real version appearing later in the user-agent string.

For example:

Opera/9.80 (Macintosh; Intel Mac OS X; U; en) Presto/2.2.15 Version/10.00

This is similar to the way all Gecko-based browsers identify themselves as Mozilla/5.0, then list the real browser name and version number later on, which makes me wonder why they didn’t just stick with that increasingly irrelevant prefix — though I suppose any scripts looking specifically for Opera versions might have still picked up Opera/10 later on in the ID.

It’ll be some time before Firefox or Safari runs into this issue, but with Internet Explorer 8 in wide release, you have to wonder…what will Microsoft do when they get to IE 10?

I recently [edit: August 2006] discovered exactly how the Wayback Machine deals with changes to robots.txt.

First, some background. I have a weblog I’ve been running since 2002, switching from B2 to WordPress and changing the permalink structure twice (with appropriate HTTP redirects each time) as nicer structures became available. Unfortunately, some spiders kept hitting the old URLs over and over again, despite the fact that they forwarded with a 301 permanent redirect to the new locations. So, foolishly, I added the old links to robots.txt to get the spiders to stop.

Flash forward to earlier this week. I’ve made a post on Slashdot, which reminds me of a review I did of Might and Magic IX nearly four years ago. I head to my blog, pull up the post… and to my horror, discover that it’s missing half a sentence at the beginning of a paragraph and I don’t remember the sense of what I originally wrote!

My backups are too recent (ironic, that), so I hit the Wayback Machine. They only have the post going back to 2004, which is still missing the chunk of text. Then I remember that the link structure was different, so I try hitting the oldest archived copies of the main page, and I’m able to pull up the summary with a link to the original location. I click on it… and I see:

Excluded by robots.txt (or words to that effect).

Now this is a page that was not blocked at the time that ia_archiver spidered it, but that was later blocked. The Wayback machine retroactively blocked access to the page based on the robots.txt content. I searched through the documentation and couldn’t determine whether the data had actually been removed or just blocked, so I decided to alter my site’s robots.txt file, fire off a request for clarification, and see what happened.

As it turns out, several days later, they unblocked the file, and I was able to restore the missing text.

In summary, the Wayback Machine will block end-users from accessing anything that is in your current robots.txt file. If you remove the restriction from your robots.txt, it will re-enable access, but only if it had archived the page in the first place.

(Originally posted as a Slashdot comment.)

I’ve been using Opera off and on for about 10 years. I think it was 1999 when a classmate showed me Opera 3.6, and how fast and small it was. (This was back when the installer fit on a floppy disk — and back when that actually made a difference.) I’ve followed it as they expanded from Windows onto Mac and Linux, onto high-end cell phones with Opera Mobile, and finally onto every Java-capable phone with Opera Mini. I’ve watched as they went from trialware to ad-supported to freeware business models. And while the desktop browser is no longer the speed demon it used to be, it’s been a consistent innovator in terms of both browser features and web capabilities.

So I’d just like to say: Happy 15th birthday, Opera! Just think, in a year, you’ll be old enough to drive!*

*In California, anyway. I think in Norway the driving age is 18.

IE6 is now two versions behind the current release.

IE6 is almost 8 years old (it was released in 2001).

IE6 is lacking in many capabilities that all other modern web browsers have, in web technology, in security, and in features you can use.

You can read a review at Wired, a write-up from the IE team, or a summary of technical changes from WaSP.

Of course, Internet Explorer isn’t the only option out there. There’s Firefox, Opera, Chrome and a host of other alternative browsers that are worth checking out.

If you’re still running Windows 2000 or some other old version of Windows that can’t run IE7 or IE8, I’d absolutely recommend Firefox or Opera. Either will be much better than IE6, both will run on Windows 2000, and Opera will even run on Windows Me and Windows 98 (but you really ought to move to something more current than Windows Me.)

I’ve also released that site under the Creative Commons Attribution-Share Alike 3.0 license, which should simplify matters for translations.

Finally, as a compromise between a full blog and little notes on the home page, I added another Twitter account, AltBrowser, where I’ll post not just site updates but random bits of news, comments, tips, etc. related to the topic.  I don’t have time to maintain yet another blog.  And I’m not convinced the net needs one.

I still hope to do that major rewrite, but this should bring it mostly up-to-date.

The open-source project started in 1999 with the goal of creating a small, fast, highly efficient graphical web browser that could run well even on low-end hardware and software. It’s a UNIX application, and runs on Linux, BSD, Solaris, etc. Things stagnated when it became clear that GTK1 was going to vanish, and GTK2 would not fit the project goals, and eventually the browser was ported to the Fast Light Toolkit (FLTK).

If you’ve used Dillo before, some of the improvements in this release are multiple character set support (the old versions were Latin-1–only), tabbed browsing, HTTP compression, anti-aliasing, improved rendering and UI, and smaller(!) memory usage.

It does have its limitations, and a few major items stand out as missing when compared to other modern browsers:

• No CSS stylesheet support.
• No scripting.
• No plug-ins.
• Limited SSL support.

That said, it’s useful to keep around on an older system, or for situations where speed is more important than rendering, or to test how a website works without styles, scripts, and plugins.

I started building RPMs of Dillo for my own use back in 2002, and became the official RPM packager for the project the following year. I’ve posted Dillo RPM packages for Fedora 9, RHEL 3, RHEL 4, and RHEL 5. Other distros will have to wait until I get my build system out of storage or figure out how to convince mock to let me build two packages together.

Of course, things have changed a lot since 2005, so I’m planning an overhaul of the whole site.

Definite plans include:

• Reflect the current state of web browser marketshare & technologies.
• List new tools, such as Dragonfly and Firebug.
• Update the list of browsers. (Add Chrome, figure out whether anyone uses Galeon anymore, etc.)
• More button/banner sizes.
• Release the whole thing under a Creative Commons license so that it’s easier for people to do translations.

I’m considering adding a blog for browser-related commentary. This would have to be low-frequency, comparable to something like css3.info, or I would have to bring other people in, because I just won’t have time to keep it updated regularly. On the downside, it adds a layer of software complexity. Also, I’m not sure it would add anything to all the blogs out there that currently focus on browser news.

I’d appreciate any suggestions from the general web user, developer, and browser fan communities.

Thanks in advance!

Good:

• Site compatibility seems to be fine so far, with a couple of minor issues (see the “Bad” section).  Mostly I’ve tested it with a couple of forum sites, LiveJournal, Slashdot, and WordPress.
• I like the simple settings box, with “Basics,” “Minor Tweaks,” and “Under the Hood.”
• It does feel fast.
• Showing the URL of links in the lower left-hand corner is a perfect compromise between the spatial advantages of a permanent status bar and the extra room provided by leaving it out.
• I like the task manager for the browser itself.  It’ll be good for developers, but it’ll also be good for users: as the comic points out, if your browser starts chewing up all available resources, you’ll be able to tell what page/plugin/program is at fault instead of just blaming the browser.

Bad:

• Gears support doesn’t seem to work quite right.  WordPress.com doesn’t detect that it’s available.  Local WP installs with Bad Behavior can’t sync completely.  (It doesn’t send an Accept header on the request for one of the TinyMCE files, which causes Bad Bahavior to think it’s a spambot and triggers a 403.)
• Cookie management is too simplistic.  I like to accept all cookies temporarily, but clear everything when I end my browsing session, with exceptions for sites where I want to stay logged in.  This is easy in Firefox, a little trickier in Opera, and doesn’t seem to be an option in Chrome.
• I have seen it pause a couple of times, with as few as 5 tabs. [edit: these seem to be related to Flash content]
• No Incomplete spell-check.
• I keep hitting the forward-slash key to search within a page, since that’s the shortcut I’m used to in Firefox and Opera.

Opera: A new Opera 9.5 preview came out today, showcasing the browser’s new look. Also, the Opera Core team takes a look at what you can do if you put hardware acceleration on the whole browser.

Internet Explorer: IE8 beta 2 is scheduled for August. I’m looking forward to seeing what they’ve done, and figure I’ll start updating sites to accommodate changes. I held off changing too much when IE8b1 came out, because some of the differences were obviously bugs (triggering the Caio Hack, for instance; and yes, I reported it).

Flock has been moving ahead with small, rapid releases, adding integration for new services each time. They just added Digg and Pownce in Flock 1.2 a few days ago. Now they’re getting ready to start on Flock 2.0, which will merge in all the new capabilities of Firefox 3. That means it’ll get new rendering capabilities, better memory management, probably EV certs and such.

Back when I wrote the original post, I had a series of 5 or 6 milestones in mind, but decided to keep it simple and only post the first two. The next one after Firefox passing IE6 was for Firefox 2+ to pass IE6. I should have been checking in more frequently, since it already has.

So what’s next? Well, I expect to see the following in the next year or two:

• Firefox 3 replacing Firefox 2. It’s already got a strong pre-release following. (Fx2 will stick around while there are still Win98 and WinMe users, but they’re already at less than 1% here and falling.)
• Firefox 1 fading into the sunset in favor of newer, more capable releases.
• Netscape disappearing into history. (It’s already below 1% here.)
• IE6 dropping below 25%, 20%, 10% (watching it go to single digits will be satisfying), and finally 1%.
• Safari approaching 10%. It’s holding steady here, but keeps climbing globally.

Things I’d like to see, but am less confident about in the near-term:

• IE6 disappearing from the radar. There are hold-outs, both at the user and the sysadmin level, plus a sizeable minority on Windows 2000. Plus I think Microsoft is committed to supporting IE6 through the lifetime of Windows XP, which means they’ll keep shipping security fixes until 2014. On the other hand, IE 5.0 is technically still supported as part of Windows 2000, but I see very few IE5 visitors these days.
• IE8 replacing IE7, for most of the same reasons it’s taking so long for IE7 to replace IE6.
• Opera breaking out of its steady marketshare and hitting a solid 5%. That would make them much harder to ignore. (10% would be better, since Safari’s still struggling for recognition at 6%.) Of course, to get there they’ll have to pull off a major publicity coup.
• IE dropping below 50%. Could be done, but it’ll be tough. If there’s no majority browser, it’ll be very difficult to justify building a site for one browser only.

Of course, these will probably all happen faster locally than globally, since the audience seems to skew slightly toward the alternatives, but then local stats are the ones that actually matter for a specific site.

Firefox and Safari, currently at just before and just after 3, are likely safe for now, but IE is creeping up on 8, and with their new, faster release schedule, IE10 may only be a couple of years away.

I’ll admit, I’ve written code like that myself (not the specific example, but I’ve done regexp matches that only look at the first digit), but always on sites that I expect to be able to maintain. Of course, one of the lessons to learn from Y2K is that shortcuts get entrenched, and code you thought you’d have time to clean up long before it became a problem has a tendency to stay in use far longer than you expected. And we’ve seen the same thing with web script archives, where someone’s example code that mostly worked in IE4 gets enshrined as “the” way to accomplish something, even though there have been better ways that work more consistently for years.

Codedread comments on Apple’s Web Inventions.

Asa Dotzler counteracts FUD about the safety of Firefox, Safari, and other alternative browsers. His main point: the key measure of security is not the number of vulnerabilities, but the window of vulnerability: the time between a hole being discovered and the patch getting onto users’ systems. (In addition to a responsive security team, automatic updates really help here.)

In just over a week, Opera’s new developer toolset, code-named Opera Dragonfly, will be ready for an alpha release. This will be a welcome addition, not just for developers, but ultimately for Opera users as well. Obviously, it’ll make it easier for web developers to debug compatibility issues, leading to fewer sites breaking in Opera. But it could also bring more people in. Firefox’s growth got started with recommendations by techies. If Dragonfly proves to be as good or better than Firebug, developers will spend more time with Opera, which could lead to recommendations.

Opera 9.5 beta
Speed, security, and performance matter.

Now, we’ve made the fastest browser in the world even faster. Opera’s new beta is quicker to start, faster at loading Web pages and better at running your favorite Web applications.

Hmm, a red and yellow blur, zooming across the view? And an emphasis on speed? That reminds me a bit of this guy:

Opera has long promoted itself on its speed, and it has used a super-hero theme in its advertising before. The vaguely Superman-like* “Opera Man” was used heavily in advertising Opera 8, despite being ridiculed by most of the browser’s user community.

So why not a subtle reference to the Flash?

*Blue costume + red cape. Hey, if a blue shirt and red jacket work for Clark on Smallville, you know the color scheme has become iconic.

Well, yeah. That’s our job.

And a bunch of random websites blocking IE6 aren’t going to convince people to change. If I were to block IE6, or only allow Firefox, or only allow Opera, I’d have to have seriously compelling content to get people to switch. Mostly, people would get annoyed and move on. Who’s going to install a new browser just so they can read the history of the Flash? Or choose an ISP? Or buy a product that they can get from another site?

Slapping the User in the Face

It’s so easy for someone to walk away from your site. One of the tenets of good web design is to make the user jump through as few hoops as possible to accomplish whatever you want him/her to do. Every hoop you add is an obstacle. Too many obstacles, and they’ll just go somewhere else more convenient.

Back when I was following Spread Firefox, every once in a while someone would suggest blocking IE. Every time, people like me would shoot it down. And think about it: what does the average Firefox user (or Opera user, for that matter) do when confronted with a site that will only run in IE? Fire off a complaint, or move on, unless it’s something they can’t live without, like, say, their bank. Only then will they bring up the site’s preferred browser…just long enough to do their business and move on.

Plus it goes against the grain of the concept that a website should be viewable in any browser. It offends my sense of… I don’t know, egalitarianism.

Recommend vs. Demand

My current tactics: I target the latest versions of each browser (or rather, the overlap in their standards support), toss in enhancements where I think something would be nice, but not critical (off-site link icons using generated content, for instance, which works in everything except IE≤7, or rounded corners, which only work in Gecko and WebKit so far). And I take that, and make it look reasonably good in IE6. I don’t try to make it perfect anymore (case in point, the header of this blog), but I try to make sure it’s functional and doesn’t look broken.

Then I include a polite notice recommending that people upgrade to something a little more capable or modern for a better experience, but I don’t require them to do so. I don’t pop up anything that moves, or blocks content, or forces them to click through an extra page.

Enter: PayPal

Now, remember what I said about banks? PayPal intends to block “unsafe” browsers from accessing their site (via Slashdot). They aren’t technically a bank, but PayPal is actually in a position where they might be able to do it: they’re the most well known online payment service where two random people can send each other money. Probably more people will switch browsers and keep PayPal than switch payment services and keep their browser.

They’ve since indicated that they don’t intend to block “current versions of any browsers,” but will focus on “obsolete browsers on outdated or unsupported operating systems.” So you IE4 users on Windows 98? Upgrade already! (And since you can’t install IE7, try Opera. It still runs on Win98!)

They’ve also cited such safety features as phishing protection (present in IE7, Firefox 2, and Opera 9) and support for Extended Validation SSL Certificates (present in IE7 and the upcoming Firefox 3 and Opera 9.5).

Hazards of Browser Sniffing

Of course, once you start actively blocking browsers, you have three choices:

• Keep track of every single browser out there, and every version.
• Let most browsers in, but only block a few problem browsers (similar to Yahoo’s Graded Browser Support)
• Unfairly block browsers that might be perfectly adequate just because you can’t be bothered to investigate them.

The last seems the most prevalent. Just ask any Opera user today, or any Firefox user of 3 years ago. (I remember using Firefox and being told to “upgrade” to Netscape 6, even though NS6 was based on an older version of the same engine. Remember: Gecko is Gecko.)

Whitelist approaches to browser detection are, by their nature, either going to require constant updating or block too much. In this case, issues would include:

• Less well-known browsers, like Flock, which uses the same anti-phishing features as Firefox
• Browsers that don’t do phishing detection themselves, using third-party plugins to do the job.
• Changes in status, when browsers add the capabilities required to get on the list.

Thankfully, it looks like PayPal is going with the most minimally-intrusive approach: blocking only the most troublesome browsers, and letting the rest connect normally.

Will it Work?

There’s still the question of whether it’ll actually make users less likely to land on a PayPal phishing site.

For one thing, it’s not clear whether they’ll block IE6. The initial report would definitely have excluded it, since it lacks both EV support and anti-phishing (without an add-on). But the follow-up statement was focused on Safari. Does PayPal consider IE6 to be a “current” version since Microsoft still supports it? Or do they consider IE7 to be current, and IE6 to be obsolete?

Certainly, if they don’t block IE6, this will really only impact the tiny fraction of users running horribly outdated software. (Well, more horribly outdated.)

The thing to remember is that the features PayPal is promoting will only help if users switch for general browsing. In fact, anti-phishing will make no difference at all on PayPal’s actual site, unless it gets hacked (at which point the user is screwed anyway.)

So let’s suppose that they do block IE6. As much as I’d like people to switch to Firefox or Opera full-time, I’m sure there will be some people who only fire up an alternative to use PayPal, and who stick with IE6 the rest of the time. They’re just as likely as before to click on a bogus “Pay with PayPal” button, or a link in a phishing email. If they weren’t going to do that in the first place, the browser requirement wasn’t needed. If they were, the browser requirement doesn’t help. The bogus sites won’t require phishing detection, or EV certs. Imagine the user saying, “Hey, PayPal fixed the problem where it wouldn’t let me use IE!”

And of course it won’t stop someone with a stolen login and password from connecting using an “approved” browser.

The ISC has also weighed in re: limitations of EV certificates. Among other things: it may be easier to get an EV cert than suggested, in which case it won’t indicate any greater degree of trust than a standard SSL certificate. And it doesn’t prevent other issues, like keyword loggers or trojans that simply hijack a user’s session.

I apologize for the rambling nature of this post (yeah, site title and all that). But I worked on it on a succession of late nights, and decided it was time to just post the thing. Also, I should have a somewhat more concise post up on OperaWatch soon now.

The key issue, of course, is moving as many users as possible from a discontinued browser—there’s no doubt that security holes will be found in it over time—to one that is actively maintained.

Why Flock, specifically? Well, sticking with the same toolkit and user profile makes migration easier, so that narrows the field to Firefox and Flock. (Not sure about SeaMonkey’s profile.) Since Netscape 8 and 9 were big on integrating with websites, Flock’s “social browser” seems a slightly better fit. And it turns out most of the Netscape 8 team went on to build Flock. Talk about social networking!

(via Flock: The Netscape Spirit Lives On)