Search Logger
Posts from: Stewart Butterfield

Author Archive

Geotagging – one day later

12:23 am - August 30, 2006 in FlickrBlog

The First Day
When we were doing our projections for how many photos Flickr members would geotag, we though that we'd hit Spiral Jetty a million in the first month, maybe even as fast as two weeks. Instead, 24 hours in, there were 1,234,384 geotagged photos (and now more than 1.6 million geotagged photos as I write this, about 9 hours later). Crazy!

(On the left is a nice example of geotagging in a magic spot - the Robert Smithson land art project, Spiral Jetty on Utah's Great Salt Lake. See whileseated's photos around Spiral Jetty.)


The Tech Behind the Scenes

One of the "little" things that was incredibly complex technically was the integration of location-based searching into our existing tag and text-based search technology. That means you can do things like search for photos matching "food" in southern Asia or architecture in South America. [We've opened up comments on this thread: please post some of your favorite searches!] This is already pretty awesome after one day — imagine a year from now. Or ten years from now. Or one hundred :)

But the reason I say it's complex is that marrying "traditional" search with spatial search in a real-time context is extremely hard, especially at our volumes and rate of growth. More than 228,000,000 photos have been uploaded, with over a million new photos being added on a good day. There are billions of bits of data that go into the search (more than half a billion tags alone), along with privacy controls, group membership, and so on.

This is one of the largest real-time search indexes in the world. In contrast, nearly all web search is done in a "batch" mode with periodic updates, while nearly all real time search is done on a small set of items whcih "expire" after a short period. But new or updated Flickr photos are typically searchable in under a minute.

Luckily, we were able to build on the amazing work of Yahoo!'s Search team. In addition to directly recruiting some of the smartest search scientists and engineers in the world, Yahoo! has rolled up a number of all-star teams through acquisitions over the years, including AltaVista, Inktomi, AlltheWeb, Overture and FAST. We get a lot of support from our brainiac brothers and sisters in the Yahoo! Search Group.

In particular, VESPA (the codename for an internal search technology that we use) enables both real-time indexing at our scale and the ability to retreive photos in any corner of the globe with any tag or text, belonging to any user or group ... and it does so fast. I'd be remiss without a special shout-out to the VESPA team: thank you!

Over time, we'll be able to apply more and more cool technology coming out of the Yahoo! Search group, along with amazing too-hot-for-prime-time projects from Yahoo! Research. (And that's going to mean a lot to you in 2012 when you have 150,000 or whatever photos on Flickr and are reminiscing back to the days before the invention of direct retinal capture technology.) I guess the acquisition worked out ;)


The Maps Themselves

There's been a lot of grumbling about the maps detail and coverage of Yahoo! maps outside the U.S. We definitely hear it, and the Yahoo! Local/Maps team, which happens to sit across from the Flickr team hears it (and feels it!) ... loud and clear. The maps in much of the world are not good enough yet. We agree.

But, fear not: maps are continually being upgraded and street-level detail for more countries has been in the works for a while already. Satellite imagery upgrades already happen a couple of times each month (the Earth is big, y'know) and in the months since the new Maps beta launched, they've already made significant improvements. More are coming.

Right now we got what we got, but the platform underlying it will automatically pick up on improvements as they are made (and in the meantime, one point in the Maps team's favor is having good labels for countries around the world: check out themexican's geotagged photos on a map. His travels around western China and central Asia are one of my favorite things captured on Flickr, and seeing the place names makes it all a little more real.

But yes: more, better, soon!


Geo APIs released

Finally, for the hackers reading this, today we're also releasing extensions to Flickr's API to enable adding and retreiving geo information, setting privacy permissions, and searching by location: everything you need to roll your own. The docs are under the link above (scroll to photos.geo) and the best place to dig in as on the developers' mailing list (see today's announcement).

This also means: "hey, if our maps don't work for you, use whatever maps you'd like!" We expect the momentum of third party geo-based Flickr "mashups" to continue and hopefully this will let developers take it to the next level since they don't have to do their own spatial indexing any more, can enable private geo-data  and get the new "accuracy" and search-within-locations features thrown in. Go for it!

And it's worth a little more props for the Search group here: we expected to have to do a few weeks of stress-testing before releasing the API since we're going a few steps beyond the cutting edge, but even with the unexpectedly strong uptake of geotagging, it's clear that the systems can handle it, so we're releasing earlier than planned.

All in all, a wicked few days for the Flickr team (thanks for your incredible work everyone!) And thanks to everyone who uses Flickr for making it even better, all the time. (Now, post your favorite map searches!)

UPDATE: Bonus link! (I just got sent this.) Yahoo! Research Berkeley's ZoneTag smart mobile upload client just got a new feature: if you take a photo "near" an Upcoming.org event (in time and space), it'll automatically get tagged with the correct Upcoming event and show up on the corresponding event page without you doing anything. Very snazzy.

 

Weekend wrap-up

11:30 pm - September 1, 2006 in FlickrBlog

A bunch of quick links for the end of this week. To those who get one, have a great long weekend!

  • QOOP recently launched myCards, standard-sized personalized business cards with your photos, single or double sided. To help get the word out, they are offering Flickr users 50% off all orders until midnight September 17th - that means an order of 48 costs only $4.99 (plus postage). Sign in to QOOP with your Flickr account to get started

  • Popular third party Flickr geotagging service Trippermap has updated its code to support our new geo APIs which means better integration and faster loading. Trippermap let's you geotag your Flickr photos using Google Earth (watch the video tutorial to see how) and subscribers get embeddable Flash travel maps with Flickr photos for their own site, as well as browsing features using Google Earth and Google Maps.

    (Trippermap was formerly known as Flickrmap, so you might know it by that name instead. Operator Mark Zeman graciously agreed to a name-change due to a trademark issues.)

  • The slightly geekier Yuan.CC set of Flickr tools, which includes GMiF (Google Maps in Flickr) and maps.yuan.cc has also updated to use the new API methods and it looks like adding location info to Flickr photos through syncing GPS tracklogs is around the corner.

  • The developer of chic-looking loc.alize.us and the handy localize bookmarklet (discussion) is also working on support for the new methods.

Finally, we'll be down for routine maintenance tonight for up to two hours, starting at 10pm paciific time (GMT-8; view in your  timezone). Back up faster and stronger right after! UPDATE: the downtime only took 44 minutes and all systems are GO!

Stenella Frontalis

Great shot of a flying Stenella Frontalis (the Atlantic spotted dolphin) by PedroMadruga. Thanks!

 

Status: Up

1:10 am - September 2, 2006 in FlickrBlog

The downtime took significantly less time than expected, so we are now back in business, 44 minutes after it started. Thanks for your patience!

Aaaaand ... the aforementioned downtime is now upon us. Of course, we apologize to anyone annoyed or frustrated by the lack of Flickrability right now, but the occasional maintenance window is still a necessity. We're moving more and more operations across multiple data centers and in the not-too-distant future, moments like these will be a thing of the past. Jolly good & carry on.

 

Link roundup

11:43 pm - January 9, 2007 in FlickrBlog

The Post-It Note Jaguar A jaguar completely covered in post-it notes as a prank (see also this Flickrcentral thread).


Gerald R. Ford Funeral | Secret Service Agent Kevin Flickr user Mr. Gillette of Grand Rapids, Michigan covered the funeral of former US President Gerald Ford as a foray into photojournalism.


Aerial elephantsMichael Poliza's fantastic aerial photos of Africa (via Kottke). So much good stuff here — and a global search for 'aerial africa' brings up a whole treasure trove.


Bonus link: Scots, check the Sunday Herald this weekend for a special feature on As We See Ourselves (some details) See also the Scotland group on Flickr.

 

It fell off the back of a truck

10:23 pm - January 22, 2007 in FlickrBlog

Leon Neal, AKA TabscoKid, a freelance news photographer and Flickr member got up at 5AM to head down to a beach in Devon to take some amazing photos of the aftermath of the MSC Napoli going down. His story below:

Worth the 5am start for this picture

"A Police Officer walking past two of the shipping containers washed up on the beach at Branscombe, Devon following the beaching of the cargo ship MSC Napoli after it was holed during storms on Sunday, 21st January 2007. The ship is carrying almost 2,400 containers, about 150 of which are said to hold hazardous chemical substances. The drifting vessel was being towed to Portland Harbour in Dorset for a salvage operation, but the Maritime and Coastguard Agency decided to beach it in Lyme Bay, near Sidmouth, instead, following the structural failure. The area is a World Heritage Site and also known as the Jurassic Coast," - Worth the 5am start for this picture, TabascoKid

There are a lot more truly spectacular shots of this amazing event in his stream:

Oops Barrel of laughs Pride Might take a bit to get it started..

 

A Key Benefit of Vista

7:35 am - January 30, 2007 in FlickrBlog

One good reason to consider an upgrade to Vista, Microsoft's just-released upgrade to Windows: the default set of desktop wallpapers it ships with include several from Flickr members. Long Zheng has a blog post with some examples, and Microsoft's Raymond Chen has more details.

Reportedly, Microsoft experience designer Jenny Lam considered around 10,000 images, combing traditional sources and commissioning a few photoshoots, but is happiest with the ones that came from Flickr members, like these from Hamad Darwish AKA [ HD ]:

Lost Sensations Destination - vol. II

 

Downtime tonight

10:13 pm - February 3, 2007 in FlickrBlog

And we're back up and running!

UPDATE@12:45pm: In a stunning reversal, it now appears that we'll be returning slightly late, rather than somewhat early. Our apologies! Current estimate is 1:40am (San Francisco time).   

UPDATE@11:50pm: Still going smoothly. Looks like we might finish up  a few minutes early.

UPDATE@ 9:40pm: Flickr's now down and busy bees are switching and racking and installing and setting up. All proceeding wel!

Tonight at 9pm (San Francisco time - view in your local time) Flickr will be down for maintenance for up to four hours. We apologize for the length of this downtime, but we're making some big changes which will enable us to roll out new features faster. So ... it'll be worth it!

As usual, photos will still be served, but you won't be able to access Flickr site or API. We'll post here once we start and when we're ready to come back up. Thank your for your patience!

 

360° of the Bois de Vincennes

10:54 am - February 5, 2007 in FlickrBlog

Back in November, both gadl and Seb Przd visited Bois de Vincennes, the large park just east of Paris:

Butte aux canons Small wood planet

Both, of course, had their cameras, and both had a penchant for hyberbolic projections of panoramic photos. We're used to coincidences around here, but when two shots like these get taken on the same day so close together and the photographers find each other .. it's still worth noting!

A bonus photo of the Bois de Vincennes, from the other end of the spectrum, unretouched black and white, from film - an amazing image by Um dia. Uma foto.:

Bois de Vincennes.

 
 
 
 
 
 
It's All About Search | © clsc.net |
2012.05.1823:23
Tech used here: Valid HTML - Valid CSS - Valid RSS - JavaScript - PHP - Smarty - MySQL - and a partridge in a pear tree.