Yvo Schaap

Recent Product Hacks

I've always enjoyed launching small product hacks. Building something from scratch gives a great rush. Hacking together API's, backend code, new functions and responsive layouts. A hackers mentality sets aside any sensible doubts which could lead to the trap of overthinking, while also challenging the brain to kick into a creative thinking and problem solving mode . Some of these hacks became timeless successes, others vanish into a folder on my PC.

Below a few recent projects I launched:

  • With the power of the Google Analytics's Real Time Reporting API I created a beautiful search activity visualization. A remake of Google's own live search presentation, but made for any site running Google Analytics. It's listed in the Google Analytics app gallery.
  • Producthunt.com leaderboard, no API so I'm crawling the site daily. Setup to get insights into whats going on in the Valley. It provides an overall overview of the activity on Producthunt, with a focus on the hunters as well as the products. I also found out I got so much data points, I can easily flag dubious voting going on. Upvote.
  • I really enjoy YouTube. But it seems they go out of their way to make it difficult to just continuously play a user channel's as one big playlist. Point in case, Majestic Casual, a video channel with over 1.7M followers, which serves a great selection of music to play in the background. So I created the unofficial majesticcasual.tv, which does what I want and looks nice too.
  • Marc Andreessen tweet essays fixes the readability of the thoughts of well-known investor Marc Andreessen which are spread out over multiple tweets on Twitter. @pmarca is known to tweet up to a dozen tweets - a tweetstorm -, but Twitter has no good way to group and read them. I even found they roll up his intermediate tweets in the stream. The project retrieved very positive high profile attention on ProductHunt and on Twitter. Really cool to see the people I follow to also mention this project on Twitter.
  • Solved a annoying issue where the native filters on iPhone photos can't be accessed on a Windows computer because the filter is only part of the meta data. The previous solution was to email myself the photo. A better solution is this crazy hack that uses the Dropbox API. I auto sync with Dropbox and this way I just select which photos I want to save back into Dropbox with the filter baked in them. Magic!
  • And my latest hack is Citytrip which collects recommendations from Airbnb hosts and lists them on a map. Great for discovering local places to eat and drink. More details on the Producthunt listing.

All of these projects scratch a personal itch, are developed in a handful of hours, and more important were fun to make and able to capture some of the web's attention.

Posted by

Producthunt.com hunters analyzed

Exploring the insiders in the startup scene

Only live to the public for 100 days, Producthunt.com is already bolstering an impressive affluent user base. Launched initially as a 20-minute MVP by Ryan Hoover (ex-PlayHaven, Startup Edition) and further developed into an actual product together with Nathan Bashaw (General Assembly), its user engagement and traction show they are on to something.

Their invite-only system has grown a user base featuring a who’s-who of the startup tech scene. Well known investors, founders, journalists, and developers are all hanging around.

My everlasting interest into the startup scene sparked me to look more close into these so-called tech insiders. A quick crawl of the site pulled in all 636 contributors, whom posted a total of 1571 products since October ’13.

By matching up their Twitter username with the Twitter API, I was able to pull in more profile details like location and Twitter reach.

Leaderboard

Instead of doing a lenghty write up of my findings, I've coded up a leaderboard that will update every 12 hours.

I ranked the most influental people who contributed at least one product on Product Hunt by their Twitter follower count:

Ashton Kutcher doer 15,674,053
Kevin Rose Partner, Google Ventures 1,447,113
Tristan Walker 2,79,690
Brad Feld Managing Director, Foundry Group 162,671
Johnny Shahidi Co-Founder, Shots 141,352
Hiten Shah Founder, KISSmetrics 141,192
MG Siegler 139,271
Nihal Fares Co-founder & Chief Product at Eventtus 100,937
Fred Oliveira Head of Product, Disruption Corp 59,428
Hunter Walk Partner, Homebrew 57,575

If we look at the site's activity, these hunters have most reach:

Ryan Hoover Product Hunt / Tradecraft 948 #77
Murat Mutlu Co-Founder, Marvelapp 653 #44
Jonno Riekwel Product designer, Jonnotie 438 #48
Dave Ambrose Venture Investor 353 #38
Kevin William David CEO,WalletKit 342 #42
Adam Kazwell Product Manager 342 #40
Robert Shedd Founder 334 #34
Geoffrey Weg Independent 313 #23
Nathan Bashaw Product Manager at General Assembly 298 #37
James Mundy Founder, Foundbite 277 #2

Follow all these hunters with this Twitter list. If there is an interest I'll add some more features to the leaderboard.

Posted by

Site search visualization using Google Analytics

Last summer Google re-launched its user search trends page. A page which features trending search keywords around the world. A cool addition is their visualization of these Google searches. While it is a great way to visualize data, it pretends the searches are happening at Google in real-time while if you dive into the code it's setup to only updates once every hour.

Next to that the Google Analytics team launched a developer API for real-time website reporting. The API allows queries on what visitors are doing on your site right now.

As a fun project I thought of combining both of these tools into one: visualizing visitor searches using the Real Time Reporting API data.

Launch project.

You can setup your own site visualization in only two steps: first authorize access to your Analytics data, select your site, and set the query parameter for your site search, usually the letter q. Click save, and see your visitor's searches appear live as a beautiful visualization .

If you don't have many visitors on your site you might be staring at a red screen for a while, do a search on your own site to see it appear. This is an video example of my visualization. Let me know if you run into any issues.

Featured on:

Posted by

Facebook values the privacy of its billion users at $4,500

Back in 2009 I found a major security exploit on both Facebook and the than popular MySpace which exposed access to a user's personal data. I reported both data leaks to the social networks, and weeks passed as I had to convince them of the major hole they left in their security. Reluctantly the leaks were closed after details appeared on Reddit's homepage. MySpace's PR quickly denied there was an issue at all (luckily a well-regarded TechCrunch reporter could confirm my Proof of Concept did work) and Facebook proposed to send me a t-shirt as thank you (which I never retrieved).

Since, website security has shown to never become fool-proof, leading to privacy breach news stories, diminishing user's trust in handing over their personal details and content. To counter act these security breaches (and the media exposure that comes with it) most internet giants (Facebook, Paypal, Google, Twitter, Github) have setup a so called whitehat security researcher program which allows for whitehat hackers to report security leaks for it to be patched and closed ("responsible disclosure"). In exchange of the disclosure and not actually exploiting the issue, there will no prosecution (!) and a finders bounty as reward. An idea initially developed in the software industry, due to a growing black market of parties buying exploits to setup botnets and whatnot detailed in this interesting The Economist article.

Facebook initiated such a program in 2011. This year, Facebook already lists 65 people who reported a confirmed vulnerability, which Facebook defines as "[a vulnerability] that could compromise the integrity of Facebook user data, circumvent the privacy protections of Facebook user data, or enable access to a system within the Facebook infrastructure". In other words, which puts user privacy at risk. For example the researchers Nir Goldshlager, Homakov and Isciurus are all filling up their own blogs detailing their numerous security exploit findings on Facebook and have become well known on the YC news homepage.

Actually, 66 people reported a vulnerability in 2013

Given my experience in 2009 (still not a proud owner of a Facebook t-shirt) and intrigued by the bold sentence on Facebook's security researcher page "There is no maximum reward" I went out and started giving Facebook's code another peak. Tracking several Facebook developer plugins, I stumbled upon a interesting Flash file used by Facebook which serves as proxy for communicating data between domains (to work around Cross-domain browser limitations). For me a clear sign to keep digging. A few hours later, by jumping a few hoops using cleverness, juggling subdomains around, and walking around a regex, I was able to load any user data or content using the user's own Facebook session, as shown in this proof of concept video: http://www.screenr.com/SWi7 [2m11]. In the video I pull in a user's private email addresses, but it could also easily access any content, including items tagged with the privacy restriction "Only Me". Yikes!

My excitement of the discovery was obviously high, given my exploit completely exposed, just like in 2009, a complete Facebook user account.

Facebook's bounty

Facebook holds an unprecedented amount of personal data and content in quantity as well as in quality, which can't be compared to any other (online) entity. There lays also one of their biggest operational risks. After finding the leak, I disclosed the details to Facebook's security team. After confirmation, my disclosure of the exploit accessing a user's account was awarded with the bounty of... $4,500. A nice day's pay, but a paltry fee for pointing out a gaping hole in the security of a social network holding the personal data of over a billion people. Without the disclosure of whitehat hackers, like I did, these exploits can also become available to dubious parties who could wreak (digital) havoc. An example could be an rogue ad network who would love to harvest and tie in the users Facebook identity for ad targeting. Facebook's PR reaction on these exploits is usualy that they haven't seen actual usage "in the wild", but obviously if it would be abused, it would be in silence (while also penal).

Even aside from whether the discovery & disclosure is worth only USD 4,500, if you hold that against the continuous struggle Facebook has with privacy of its users, the costs of its thousands highly skilled employees, and the real implications of exposing user's data to actual dubious parties, $4,500 is clearly a small sum for the help in protecting Facebook's users personal data. With 66 exploit disclosures in the past 5 months you can wonder how many exploits are found but not disclosed to Facebook, and also wonder why Facebook is so dependent on external security research & disclosure for its user data security.

Update: /r/netsec lists some more technical details if you are interested and Hacker News has an interesting discussion.

Update #2: I'm now also listed on facebook.com/whitehat.

Posted by

TechCrunch Crunched - Exposing the lastest tech trends

6 fictitious questions finally answered

TechCrunch has always been the most authoritative news outlet on whats hot in Silicon Valley regarding start-up trends, early & late investments, product launches, celeb founders and of course geeky drama. Although some frequently praised start-ups were eventually exposed as fads (Badgeville, Groupon) other were expertly picked early on (Twitter, AirBnb).

Giving it's again that time of the year for end-of-year re-captions, I've setup a research focused on analyzing TechCrunch's editorial posts, with as aim to expose this year's trends in tech. And while I was at it, I even went all the way back into the archives from when TechCrunch's was launched.

I analyzed all 106,664 posts made from January 2006 onward, looking for interesting data hidden in the individual posts.

Read my complete guest post on TechCrunch.

Posted by

Facebook Open Graph: going beyond the ‘like’

A developer’s review

Facebook is everywhere

At Facebook’s developer conference F8, at the end of last year, Facebook announced a major extension to the Open Graph. The extension allowed Facebook apps to publish arbitrary actions a user performed to the user’s account. In aggregate these actions tell a narrative about a user’s interaction with the app, whether ‘installed’ on Facebook or on any external property.

Although the Facebook API is one of the most well known and used APIs, there is little data on the actual performance of the platform when tapped into Facebook’s social reach. As developer you have to do with quotes like "Six months ago months ago Google provided 40 per cent of the Guardian’s traffic. The launch of the Facebook app resulted in a 'seismic shift' with social exceeding search as a driver on several occasions in February." by the Director of Digital Development at The Guardian.

But how do custom Open Graph actions actually perform? Does it (also?) work for non-brand-owning developers? I've done an experiment by creating an app around custom Open Graph actions, pushing traffic to the app and sharing the results with you in this post.

The Graph Update: Add to Timeline

The recent Open Graph update is best explained by example. When activated Yahoo News publish every article read action to the Open Graph. In your stream you get a article suggestions based on your collective Facebook friend Yahoo News read activity. Users don't have to perform any explicit share-to-Facebook actions, because their activity on Yahoo News - reading an article, adding a comment - is pushed to the Open Graph by the app dubbed as frictionless sharing. For Yahoo this stream of actions brings continuous exposure of Yahoo News articles on Facebook.

More revolutionary is that the app is able to control how this stream of independent actions are grouped and displayed in the Timeline as so called aggregates. These aggregates are able to add more value for a user then the independent actions by providing a (visual) grouped summary. Examples of the possibilities of these aggregates are on the Facebook Timelime introduction. These aggregates nestle themselves in the timelime of a user's Facebook profile.

Facebook's move towards canalizing this constant sharing of activity by apps is interesting. Facebook has been battling apps who aggressively pushed their promotional messages to a user’s stream since the beginning of the "allow this app to post on my behave" permission. A well known example are the constant farm updates in your stream (by your friends in exchange for a few extra coins). Instead of fighting what seemed a never ending battle against the user’s behavior exploited by apps, Facebook’s Open Graph adopts this stream of activity and gives it its own framework and place in the Timeline, Stream and Ticker (the sidebar).

The integration by major brands (currently mostly actions: read, watch & listen) has been enormous given the period of time this has been launched. These brands got behind Facebook’s proposal which of course involves tapping into Facebook’s 845 million eyeballs and their interaction on the platform, but maybe more importantly: a user's app activity exposed to their friends. Most of the integrations have been hooked into their own property, which makes Facebook even more present outside of its own walls. The push these brands are making with Facebook's “Add to Timeline” shows either brilliant business development, or a real opportunity for growth.

“Developers, developers, developers”

My experience with Facebook as a developer has been one of caution. Facebook has been notorious of changing the rules during the game, and even tackling their once star players. A recent example is the move away from Page tabs. Where until recently Rihanna’s Facebook page was able to set a default landing tab app for users visiting her profile, now tab apps have to do with a meager square linked image in the Page’s header. And while the tab used to be integrated into the Page’s branded environment, it’s got stripped down to a blank page killing the user experience. I expect no start-up to touch Facebook tab development from now on. Existing tab apps like start-up RootMusic’s Bandpage see their usage fall of a cliff. While I understand the arguments for most changes, it makes me careful to invest in the API development learning curve.

In spite of that, I wanted to try it out the possibilities with one of my sites.

I “like” to “vote”

Facebook actions have always been limited to liking objects. Now apps are able to tie any action on any object, and I wanted to make use of that. One of my sites publishes a weekly music chart. I used to have Facebook like buttons for people to vote on chart entries. Users recognize the like button, it doesn’t need registration, works inline and it gives a valuable promotion in the user’s stream.

Going from this point I wanted to move to a real chart entry vote action: [user] voted for [chart entry] which would be published in Stream, Ticker and Timeline using the Open Graph. Going to the drawing board (and back a few times) I wanted to optimize the usage of activity aggregates. Aggregates get prominent placement in the Timeline, and I wanted to explore the boundaries of what is possible with my custom Open Graph action. This eventually led to the idea of having users publish their own music chart in their timeline. The user’s timeline would also show a visual narrative of the user’s favorite music at that point in time by showing a gallery of chart entry images.
Facebook developed a extensive admin interface to create objects, actions and aggregates. They added the ability to publish sample data activity through a user interface limiting the hassle with testing different parameters. Obviously much time has been put into creating this, while still taking much trial and error to explore what the platform limitations are.

One barrier is the custom action review process. While creating an action is not the issue (vote in this case), the action needs approval before you can go live outside of your development environment. This unnecessary lengthens the phase where you hack something together to try out whether something works (or not).
Subsequent steps involves adding Open Graph meta tags to pages which describe the custom objects, and hooking it all up with the Javascript SDK.

The actions the user performs in my app result in:

Timeline Aggregate

Stream

Ticker

The Timeline shows the music chart the user published through the app, most voted for artists and recently voted for chart entries.

And the results are in…

After too much coffee & code, I pushed out a promotional message to my most trafficked pages reaching around 200k/day. The message asked user to vote for the music charts, which forwarded to the Facebook app page with a ‘Authenticated Referrals’ roadblock asking for permissions to publish vote actions.

From the stats (Hooray for Facebook providing insightful data!) I found 37% bounced back on the Facebook roadblock asking the user publish vote permission. Also the number of votes (Open Graph actions) where hanging around a meager 200 after 38 hours. Ignoring Facebook’s best-practice advice I wanted to show the content first to reduce the bounce rate, and only request permissions after a user action. All I would need to do is convince the user to vote, and he will allow permission.

The critical questions is, does this really engages a user's friends to check out the app? This viral effect should be exposed by the click through rates:

Actions get significant exposure. One action is 35x shown to friends. The CTR of 1.4% shows the action is not really good in grabbing a user's friend referral, so we don't see much traffic coming from this. This shows there is a big room of improvement on that front. The Timeline provides the biggest opportunity for traffic (1.5% against feed referrals of 0.7%). After removing the permission roadblock I don't see an improvement in the bounce percentage, but a drop in accepted app permissions.

(apologies for promising hard data, but I expect to get a larger result sample when I improve my app and promote it more)

Conclusion

I find the "My Chart" aggregate to most interesting aspect of the Open Graph capabilities. Users are able to publish their own music chart to their Timeline which adds entertaining and informational value. Unfortunately I wasn't able to explain the drag-and-drop track entries in this field to create your chart clearly so it was mostly unused in the trial period.

I can't claim the 4x Facebook referral traffic Foodspot is, but a majority of the blame probably lies with my basic app implementation. Compared to the previous Facebook like implementation I definitely got a larger number of “likes” on chart entries. But I presume Facebook will tighten the screws of exposure of standard likes, and force apps to move towards Open Graph actions. Also the vote has more of a custom branded user experience which makes it attractive to build a brand with.

I will keep on developing the app, and keep you updated on data I collect in the progress.

Update: Great minds think a like "Making use of a timeline app for voting is innovative, said Justin Osofsky, director of platform partnerships at Facebook." [source]

Posted by

More hints of Facebook Music (code leaks)

The rumor mill of Facebook finally doing something with actual Music has been going on for months. Most Facebook app developers have been steering away from developing anything related to music, afraid of being squashed by Facebook, hence the platform doesn't have any real mainstream music services.

Most of the rumors involve having the major streaming music players (pun intended) Spotify, MOG, Rdio more deeply integrated into the Facebook platform. GigaOm summarized Facebook's proposal to the music services (the mentioned Ticker has rolled out in a new design since).

22 September seems to be the date of the upcoming Facebook music announcement at f8 given the first talk beeing about "The Future of Digital Music".

Suprisingly this week MOG and Rdio both announcement support of limited free streaming, something Spotify has been offering in Europe, and more recently also in the US.

The Hints

I looked at the major music streaming services, and found an interesting reference in their HTML code. All track, album and artist pages got meta data in a yet undocumented format:

The providers whom all serve this custom Facebook format by tagging their pages music.song, music.album are:

These seem to be the launching partners. Confirmed notable services without the format:

  • iTunes
  • MySpace
  • Pandora
  • Turntable.fm
  • Amazon.com (store and cloud player)
  • Last.fm
  • Napster
  • Kazaa
  • Groovershark
  • emusic
  • OVI Music

Facebook has been building a social graph for almost anything and anyone. These services providing detailed meta data regarding music, which users could like, share and comment would be a big win for Facebook and other developers to build from. The social graph would be expanded by a detailed music profile of users, and their friends.

The undocumented mentioned audio type audio/vnd.facebook.bridge seems to refer to a format that bridges audio between the streaming services and the Facebook platform.

It seems all the partners are ready: free streaming, link between the music service and Facebook, all we need is to wait few more days.

Update: the official partner list

As featured on:

Posted by

Easy Share buttons for publishers

I’ve always found it overly complicating to attach the famous Facebook Like, Twitter Tweet and most recently the Google +1 buttons on appropriate locations. Adding these buttons with the provider provided code involves adding blocking javascript which could cause your whole site to not load at all, significantly increase total page load, lead to unwanted javascript errors and could break the existing layout. Next to those issues: positioning the buttons, adding/removing providers and creating a common layout between the buttons is a huge hassle.
While not implementing these buttons is of course an option, the push from search engines to promote retweeted links, likes & +1-ed pages of friends in their search results could harm your competitive position.

For DirectLyrics I recently created a one-stop solution which solves all the issues these buttons cause. Key issues that needed solving: quick & fast loads (2kb gzipped), wide provider support (5), and non-blocking.

Result:

  • Easy install on any page by including an iframe which refers to a CDN hosted, long term cached html file.
  • Control variables on layout, supported providers, relevant url. Passed as #hash (keeps the file completely cached for users).
  • Support both (native) tall counters as well the wide layout including the counters.
  • Providers: Facebook (Link and Send), Twitter, LinkedIn, StumbleUpon and Google +1.
  • Non-load blocking by using smart async javascript loaders.

This is an early and quick release, but I’m planning on supporting this for a while. You can download the source, or hotlink to the CDN version.

Setting

Example iframe:

 

Implementation settings go here

Sample

Posted by

The Google 1 Button Discovered

Google just recently introduced the +1 Button to move towards a more social search. The +1 button is added to each search result on Google and allows users to share what results they liked +1’d. User’s friends can see the +1 in their search results and hence act on this piece of recommendation. These +1 are also used as lists to spice up the otherwise boring Google profile pages and to influence the search results rankings.

Google users must reside in the US, have a Google Profile, be logged in and enable +1 to see how it works.

Whether the +1 is useful or not is too early to call. But a noteworthy feature is missing: a button which can be attached on news posts to let visitors +1 their content. Just like when Google Buzz was launched an ugly hack was needed for months until an official buzz button was made available. The buttons does exists because there is personalisation option available referring to non-Google sites.

Google claims the button is “coming soon” but I couldn’t wait, so I looked around the code, and looked some more, until I found the button endpoint hiding from me, obfuscated, in a stray piece of javascript.

Check out these live Google +1 buttons:
[since this was posted the button is available to anyone]

as seen on Fanity integrated in the right-hand side bar.

You can make them horizontal or vertical just like Twitter retweet and Facebook like.
[since this was posted the button is available to anyone]

What I found out:

  • They work! Clicking the button (try: Fanity) indeed makes your +1 link appear on your Google profile.
  • If the button is red with an exclamation mark you are not logged in or from outside the US. Check out the screen cap below.
  • These buttons also reveal the total number of +1’s by changing the request URL. For example Google.com has 982 +1s, Techcrunch.com 241, Reddit.com 125. Whether this is a total count from my friend-circle only I’m not sure, but it should since that would make more sense.
  • Google needs some more A/B-testing on alignment of the total +1 count.

An image for when Google takes it down, or you are not from the US/logged in:

As seen on:

  • Techcrunch
  • Business Insider
  • Posted by

    Sierre, 18 March 2011

    image
    Posted by

    1 2 3 4