Yahoo Pipes

source: https://retool.com/pipes

A tool for connecting data across the web that never came to be

• started off as a “skunkworks” effort within Yahoo as part of cutting edge ideas
  • finished version was intended for end users to help program websites
• Final product was a cloud-hosted service where users could drag and drop elements to “roll their own information feeds”
  • these feeds could take data directly from websites, be filtered, merged, and processed and produce a final stream that could be subscribed to
• Pasha Sadri lead this effort
• The problem emerged when they launched they got 100,000s of users rather than thousands and each pipe in aggregate pulled a ton of data and required a lot of processing
• 
  • looks a lot like natto.dev
• Supported RSS, Atom, CSS, RDF, etc. and eventually YQL
• and very powerful output formatting:

  • Pipes output could be embedded into a page or formatted as HTML, JSON, KML, RSS, or XML. There was even a way for a non-Yahoo website to include JavaScript that queried a Pipe workflow and received the results back without any front-end browser or back-end server involvement. Pipes, in effect, could power other sites.

• But other websites also started blocking pipes
  • ex: craigslist, supposedly because of resource hogging after someone created a map of craigslist listings (source)
    • lol they literally hardcoded “pipes.yahoo.com” and blocked it
    • but they brought it back 2 weeks later
• HN discussion on the article
  • effectively allowed you to bypass cors to build clients that used data from other sites

• A youtube video with title that changes with the view count: https://www.youtube.com/watch?v=BxV14h0kFs0