Computers get smarter, CAPTCHAs get harder

This week’s CBC tech column (CBC.ca version) is all about CAPTCHAs, and how some of them are getting harder. CUE OMINOUS VOICE. The robots are getting smarter.

[Audio to follow]

===

Tell me if this sounds familiar: you’re online, ready to buy some concert tickets or to sign up for a new email account. But before you’re allowed to proceed, you have to prove you’re a human being by deciphering a mess of distorted, squiggly letters and numbers, then typing them into a text box. This is what’s called a CAPTCHA, or completely automated public Turing test to tell computers and humans apart.

For a while now, I’ve had a sneaking suspicion that CAPTCHAs are getting harder. Increasingly, I’m left wondering, Is that an uppercase “x” or a lowercase “x”? An “o” or a zero? A “q” or an “o” with a squiggle through it? Sometimes, even though I’m 100 per cent sure I’ve typed exactly the right thing, the computer disagrees with me.

For months, I thought I was alone in this frustration. I worried that my increasing inability to pass these tests suggested that I’m not entirely human.

Then last week, I opened an email message from a colleague that read: “You know those distorted letters we have to type to pass security tests online? I notice they’re getting more and more distorted.”

According to Luis von Ahn, one of the computer scientists who coined the term “CAPTCHA,” the tests are getting harder.

“The thing about CAPTCHAs is that many people do their own implementations,” he told me. “Over time, some of these implementations have gotten a lot harder, because the really easy ones – essentially, the undistorted ones – can be broken by bots.”

Traditionally, identifying squiggly, distorted letters has been difficult for computers but comparatively easy for humans. But computers are getting better and better at it, and easy CAPTCHAs aren’t as effective as they once were.

Still, von Ahn says his own implementation of CAPTCHAs, called reCAPTCHA , isn’t getting any harder.

“It’s still the case, as it was three or four years ago, that a person who submits a solution [to reCAPTCHA] is going to be correct 96 per cent of the time,” von Ahn said. “That number remains the same.”

ReCAPTCHA, which was acquired by Google in 2009, generates more than 100 million CAPTCHA images a day for various websites for free. The CAPTCHA images it provides are also used to help decipher words that can’t be identified during the process of digitizating printed material.

Computers are getting better at solving CAPTCHAs because devising automated ways of bipassing the test is potentially lucrative. Imagine that you’re an email spammer. Wouldn’t it be great if you could automatically sign up for hundreds or thousands of bogus email accounts? Or, imagine you’re a ticket scalper. Wouldn’t it be terrific if you could write a computer program to automatically buy all the tickets for a concert? CAPTCHAs can help keep spammers and scalpers at bay.

Because there’s a lot of money to be made, software developers are actively writing code they say can crack CAPTCHAs that, von Ahn says, sells for $10,000. Von Ahn said he has even seen ticket scalpers advertise software they say can break reCAPTCHA for as high as $50,000.

According to von Ahn, it’s simply a matter of time before software will rival humans at solving CAPTCHAs, but it could take decades.

In the meantime, as easier, ineffective systems are phased out, people will continue to be frustrated by some CAPTCHAs. And as frustrating as CAPTCHAs are for the average user, they can be even more frustrating for people who are visually impaired or use screen-reader software. Some CAPTCHA implementations include an audio alternative, but accessibility will continue to be an issue.

Regardless of how they are implemented, CAPTCHAs are all built around the idea of creating a task that’s hard for computers and easy for humans. As computers get better and better at reading squiggly letters, we may be asked to prove our humanity by performing other types of tasks.

For instance, computers are still very bad at determining the contents of a photograph. It’s difficult for software to tell the difference between a photo of a cat and a photo of a dog. Microsoft Research built a CAPTCHA system called ASIRRA (Animal Species Image Recognition for Restricting Access) based on this idea. Companies like Solve Media and NuCaptcha have put their own twists on CAPTCHAs that require users to enter words from a text or video advertisement.

If von Ahn is right and computers will eventually be able to reliably solve text-based CAPTCHAs, that’s not necessarily a bad thing. Though CAPTCHA-busting technology could be used by spammers or ticket scalpers, it could also help decipher hard-to-read parts of digitized books or identify skewed and distorted text in photographs.

So, the next time you’re confounded by a mess of squiggly, distorted letters, don’t be too hard on yourself. Maybe it’s the CAPTCHA’s fault.

As von Ahn told me, “Sometimes, they’re really bad. Sometimes, they are so hard to read that I can’t read them myself.”

Comforting words from one of the people responsible for all those squiggly letters.

What exactly is a “broadcaster” anyway?

My CBC Radio column this week is all about whether the CRTC should consider streaming video services like Netflix to be “broadcasters.” It’s posted below, for posterity. Audio to follow.

===

Let’s play a little game. One of these things is not like the others: CBC, CTV, Global, Netflix.

If you guessed Netflix, you win. In the eyes of Canada’s telecommunications authority, the CRTC, Netflix isn’t considered a “broadcaster.” But some industry groups think it should be.

Part of being a broadcaster in Canada means that you’re obliged to give something back to Canadian programming. It’s right there in the Canadian Broadcasting Act: “each element of the Canadian broadcasting system shall contribute in an appropriate manner to the creation and presentation of Canadian programming.” Practically speaking, this means that Canadian broadcasters are required to give a percentage of their annual revenues to organizations like the Canadian Media Fund (correction: Canada Media Fund) — which help pay for new Canadian content. Basically, if you want to run a TV station, you need to help pay for new Canadian TV. Broadcasters have to do this.

But here’s the thing: the CRTC doesn’t consider services like Netflix to be broadcasters, so they can distribute movies and TV shows without the same requirement to help fund new Canadian productions.

Late last week, a group of Canadian film and television companies got together during an industry conference to talk about this. In addition to a panel discussion that was part of the official conference program, there was also a separate, private meeting, organized by the Canadian Media Production Association. The CBC reported that it included cable companies, broadcasters and union representatives, and the topic of conversation was essentially, “What do we do about over-the-top (OTT) services like Netflix?”

The same day, the chairman of the CRTC, Konrad von Finkenstein was quoted as saying, “It’s not broadcasting, within the Broadcasting Act.” I clarified this with a representative from the CRTC, who told me that companies like Netflix fall under the CRTC’s new media exemption.

Back in 1999, the CRTC said, “new media broadcasting undertakings are not subject to licensing by the Commission.” In 2009, they said basically the same thing: “While broadcasting in new media is growing in importance, we do not believe that regulatory intervention is necessary at this time. We found that the Internet and mobile services are acting in a complementary fashion to the traditional broadcasting system.”

That was, of course, more than a year before Netflix started streaming movies and TV shows to Canadians. Now, in 2011, I’m not so sure that “complementary” is the right word to describe services like Netflix in relation to traditional broadcasters. “Competitive” is probably more accurate.

So, who cares, aside from traditional broadcasters? Well, if you’re concerned about the amount of homegrown content that’s produced in this country, you might care about this. Because over-the-top services like Netflix aren’t obliged to financially contribute towards new Canadian production. Or, if you’re a Netflix customer, you might care about this. Because if the CRTC were to decide that it should regulate these services, and treat them more like traditional broadcasters, we’d probably see prices go up.

But for me, the biggest reason Canadians should care the precedent it could set. Netflix is just the thin edge of the wedge. What happens (or doesn’t happen) with Netflix will affect whether we see other services like Hulu, Google TV, or Amazon Video on Demand north of the border. The big issue for me is consumer choice.

But the bigger issue here is the role of the broadcaster in 2011. What exactly does “broadcaster” mean anymore? When I contacted Netflix, a representative was very clear, saying: “Netflix is a distributor of TV show and movies, not a broadcaster.”

But I wonder: in the minds of consumers, is that a distinction without a difference?

For instance, I can pay for movies on demand from the cable company, and I can pay for movies on demand from Netflix. They both travel into my home through the same wire. They both play back on my television set. But one’s a broadcaster, and the other’s not. What’s the difference, really?

And if Canada does decide to regulate online broadcasters, where do we draw the line? If we decide that Netflix should be considered a broadcaster, does that mean YouTube should be considered a broadcaster? If I take a video on my phone and email it to you, does that make me a broadcaster? Regardless of whether the industry likes it or not, or if the regulators acknowledge it, consumers’ ideas about “broadcasting” are changing.

In its 2009 new media decision, the CRTC said, “Given the pace of change in the new media environment, and in accordance with established policy, the Commission expects to conduct the next review of the broadcasting in new media environment within five years, or at such time as events dictate.” That means another review by summer 2014.

I can only hope that the Broadcasting Act doesn’t fall even father out of step with reality by then.

Back to the Future

I’m a sucker for nostalgia, as evidenced by GRTTWaK. So, you can image my delight when Jenna sent me a link to a series of photographs by Irina Werning called BACK TO THE FUTURE:

I love old photos. I admit being a nosey photographer. As soon as I step into someone else’s house, I start sniffing for them. Most of us are fascinated by their retro look but to me, it’s imagining how people would feel and look like if they were to reenact them today… A few months ago, I decided to actually do this. So, with my camera, I started inviting people to go back to their future.

Amazing.

Google algorithms use public usernames to link profiles

My CBC tech column this week is about new research from the French National Institute of Computer Science that looks at how using the same public usernames across multiple websites could lead to unwanted online profiling. The point of the research was to demonstrate that username profiling techniques are possible.

Well, now they’re out in the wild, on a large scale. Just this morning, Google announced some changes to its Social Search features, including username-linking:

if our algorithms find a public account that might be yours (for example, because the usernames are the same), we may invite you to connect your accounts right on the search results page and in your Google Account settings.

Google’s offering is optional, but it’s not hard to see how this technique could be used to link your online profiles without your consent. As I said in the column, “bad news for ‘EngelbertHumperdinck1936,’ but good news for anyone named John Smith.”

Live radio + realtime social media

This afternoon, I had the pleasure of doing a short talk on “social media” for many of the hosts and producers of local and regional CBC Radio afternoon shows. I’ve talked about social media and radio before, mostly in the context of Spark.

But here’s the thing: local and regional afternoon shows are completely different from Spark. Spark is a one-hour, pre-taped, weekly show. By comparison, local and regional afternoon shows are several hours long, and live, each and every single day of the week. I’ve worked on these shows, and they can be gruelling. So, the focus of my talk was how shows could prioritize their online efforts, given limited time and resources.

Here’s the theory: some online tools work particularly well in a realtime context, and some online tools work particularly well in an on-demand, time-shifty, random access context. The trick is choosing your tools wisely.

Realtime tools embrace what Nick Carr calls “Nowness.” What’s going on right this minute. Twitter is a realtime tool. Facebook is, in many modes, a realtime tool. Foursquare, Gowalla, and their ilk are realtime tools. On the other hand, podcasts are on-demand, time-shifty tools. A radio show website, updated once a day, is an on-demand, timy-shifty tool.

So, let’s map these tools onto radio.

Radio is “the immediate medium.” It’s fast. Live radio is full of “nowness.” Something big can happen, and you talk about it on the air right now. But live radio items can also have a long-ish shelf life. A feature interview with the mayor will still be interesting in a week, whereas your traffic and weather reports won’t.

So here’s the point I tried to make this afternoon: if, like most CBC radio afternoon shows, you have limited time and resources, be smart about which online tools you use. Use realtime tools for realtime information. Use time-shifty tools for time-shifty information.

I think realtime tools and live radio go really, really well together, and there are some terrific examples out there. CBC’s Here and Now, for example, or The Dave Ramsey Show. Tools like Twitter and Facebook are great for discussion, feedback, contests, questions, and overall, providing an additional layer on top of a live radio show.

Live radio show audiences (if that’s what we call them anymore) are passionate. They are intelligent. They want to help. They want to connect. If you ask me, as a public broadcaster, we have a responsibility to connect with them in as many ways as we can muster.

Postscript

As part of my talk, I tweeted:

Meeting w/ local @CBCRadio shows from across Canada right now. Tell me, what do they need to know about Twitter + live radio? ^dm

The responses are here and here, and worth reading.

[read now]