Monthly Archives: April 2012

People playing with collections #14: collection data on Many Eyes

Many Eyes Website

Many Eyes Website

I love seeing examples of uses of our collection metadata in the wild. bartdavis has uploaded our data to Many Eyes and created a few visualizations.

I found it interesting to see how many “matchsafes” we have in the collection, as you can easily see in the “color blindness test” inspired bubble chart! Here are a few screen grabs, but check them out for yourself at https://www-958.ibm.com.

Of interest to us, too, is that these visualisations are only possible because we released the collection data as a single dump. If we had, like many museums, only provided an API, this would not have been possible (or at least been much more difficult) to do.

Bubble chart of object types

Bubble chart of object types

Number of objects by century

Number of objects by century

Word cloud of object types

Word cloud of object types

Totally cached out

We do a good deal of cacheing on our web properties here at Cooper-Hewitt.

Our web host, PHPFog adds a layer of cacheing for free known as Varnish cache. Varnish Cache sits in front of our web servers and performs what is known as reverse proxy cacheing. This type of cacheing is incredibly important as it adds the ability to quickly serve cached files to users on the Internet vs. continually recreating dynamic web-pages by making calls into the database.

For static assets such as images, javascripts, and css files, we turn to Amazon’s CloudFront CDN. This type of technology ( which I’ve mentioned in a number of other posts here ) places these static assets on a distributed network of “edge” locations around the world, allowing quicker access to these assets geographically speaking, and as well, it removes a good deal of burden from our application servers.

However, to go a bit further, we thought of utilizing memcache. Memcache is an in-memory database key-value type cacheing application. It helps to speed up calls to the database by storing as much of that information in memory as possible. This has been proven to be extremely effective across many gigantic, database intensive websites like Facebook, Twitter, Tumblr, and Pinterest ( to name just a few ). Check this interesting post on scaling memcached at Facebook.

To get started with memcache I turned to Amazon’s Elasticache offering. Elasticache is essentially a managed memcache server. It allows you to spin up a memcache cluster in a minute or two, and is super easy to use. In fact, you could easily provision a terabyte of memcache in the same amount of time. There is no installation, configuration or maintenance to worry about. Once your memcache cluster is up and running you can easily add or remove nodes, scaling as your needs change on a nearly real-time basis.

Check this video for a more in-depth explanation.

Elasticache also works very nicely with our servers at PHPFog as they are all built on Amazon EC2, and are in fact in the same data center. To get the whole thing working with our www.cooperhewitt.org blog, I had to do the following.

  1. Create a security group. In order for PHPFog to talk to your own Elasticache cluster, you have to create a security group that contains PHPFog’s AWS ID. There is documentation on the PHPFog website on how to do this for use with an Amazon RDS server, and the same steps apply for Elasticache.
  2. Provision an Elasticache cluster. I chose to start with a single node, m1.large instance which gives me about 7.5 Gig of RAM to work with at $0.36 an hour per node. I can always add more nodes in the future if I want, and I can even roll down to a smaller instance size by simply creating another cluster.
  3. Let things simmer for a minute. It takes a minute or two for your cluster to initialize.
  4. On WordPress install the W3TC plugin. This plugin allows you to connect up your Elasticache server, and as well offers tons of configurable options for use with things like a CloudFront CDN and more. Its a must have! If you are on Drupal or some other CMS. there are similar modules that achieve the same result.
  5. In W3TC enable whatever types of cacheing you wish to do and set the cache type to memcache. In my case, I chose page cache, minify cache, database cache, and object cache, all of which work with memcache. Additionally I set up our CloudFront CDN from within this same plugin.
  6. In each cache types config page, set your memcache endpoint to the one given by your AWS control panel. If you have multiple nodes, you will have to copy and paste them all into each of these spaces. There is a test button you can hit to make sure your installation is communicating with your memcache server.

That last bit is interesting. You can have multiple clusters with multiple nodes serving as cache servers for a number of different purposes. You can also use the same cache cluster for multiple sites, so long as they are all reachable via your security group settings.

Once everything is configured and working you can log out and let the cacheing being. It helps to click through the site to allow the cache to build up, but this will happen automatically if your site gets a decent amount of traffic. In the AWS control panel you can check on your cache cluster in the CloudWatch tab where you can keep track of how much memory and cpu is being utilized at any given time. You can also set up alerts so that if you run out of cache, you get notified so you can easily add some nodes.

We hope to employ this same cacheing cluster on our main cooperhewitt.org website, as well as a number of our other web properties in the near future.

Thoughts on Skillshare's Penny Conference

Last week I went to Skillshare’s Penny Conference about re-imagining education for the 21st century. Here are my reactions to the some of the ideas that caught my interest:

Michael Karnjanaprakorn, founding CEO of Skillshare, opened the day with a talk that drew a distinction between learning and education. His company is built on the idea that in the information age, people don’t value a highly-trained expert lecturing at the front of a room as much as they used to. Education today is de-centralizing and democratizing. His triumphant tone here reminded me of Robert Wong at Bill’s Design Talks. Robert was really excited about the widening availability of digital design and media tools and their democratizing effect on culture.

Skillshare is an organization whose mission is to transform education by empowering teaching and democratizing learning.

A sense of triumph was also in the air when various speakers brought up themes of intellectual rebelliousness and dropping out of college. Rote learning, formulaic testing, and traditional metrics of success were denounced as the enemies of innovators and dreamers everywhere. These ideas uphold a lot of 20th century American lore surrounding the lives of entrepreneurial and creative luminaries. Our culture loves the stories surrounding famous dropouts like Steve Jobs and Mark Zuckerberg. I find these legends to have a strong Modernist aroma, distinctly un-21st century– the romance of a great lone genius. My hope for education in the 21st century is that our culture will put very high value on humility & interdisciplinary teamwork as weapons against big complex problems. A good team, I think, would have a healthy blend of rule-following experts and rebellious thinkers. Actually, I think both traits can coexist in one person. It shouldn’t matter whether you’re a dropout or a PhD– what matters is what you bring to the table.

Baratunde Thurston spoke about something that interested me– the idea of a porous, infinite book. He talked about his unique process of writing How To Be Black, the coolest part of which was the idea of “live-writing.” Live-writing entails publicizing a web link where anyone may view a broadcast of the author’s computer screen as he works. I think this is a really cool idea. Anybody who works on a computer could try this “open process”–a video editor, a graphic designer, an architect. Baratunde said he did not pay attention to the comment stream as he worked, however he did review the comments at the close of a writing session. I like how this, as a gesture, takes some of the mystique out of creative work. Now that the book is printed, he uses Twitter provocations and other web means to keep the conversation flowing and open. He views the final book as porous– not a finite, bound object. In the 21st century, printing no longer implies that a book is complete. I like this infinity idea because it feels truer to the way people really interact with a text–always citing, discussing, debating, revising.

Zach Sims of Codecademy argues that in the 21st century, “understanding algorithms” ought to be added to the core skills of Reading, Writing and Arithmetic.

 

I liked what Charles Best had to say about “pushing intelligence out to the edge.” The phrase comes from the security industry, referring to advanced device networks where a non-central node in the network can make intelligent decisions without querying the central node. Charles is excited about web platforms like Kickstarter, Etsy and his own website, Donors Choose, because they jettison the traditional gatekeepers and middlemen of cultural production, pushing [cultural] intelligence out to the public, which allows good ideas to come to life more easily and quickly. Charles pointed out another great outcome of pushing intelligence out to the edge, which is that “solutions can come from the front lines.” Many good examples of this phenomenon are included in our Design With The Other 90% exhibition series. The 90% series features lots of design solutions for different problems that originate from the end users themselves. There’s no need for products and services to come down from on high (governments, large companies, powerful institutions) any more. If you are interested in this topic, check out the Social Impact Design Summit we held early this year.

In sum, decentralization of ideas, learning, and teaching was the major theme of the day.

I thought the day could have been even better if someone had discussed an important group of concepts: prestige, class & social mobility. If we’re really going to re-imagine education for the 21st century in an inclusive, sweeping way (which was, I think, what the conference organizers intended) it’s important to keep aware of the different meanings “education” has for all. There’s a powerful association in our culture between education and social mobility. Adam Braun’s talk about building schools in Laos, Nicaragua and Guatemala did introduce the idea of access, but nobody made the more challenging mindset leap–it’s the shorter leap–to the complicated, insidious disparities in our own city. The opportunity to affiliate oneself with the status and high regard of a respected institution is a privilege. I think it’s a mistake to excessively glamorize an informal educational ethos without acknowledging the reality that going out on a rebellious limb is most comfortable for those with an economic safety net. Hacking education is great, but the traditional currencies of prestige and status can’t be omitted from a conversation about transforming education for all.