Free community accounts on the ScraperWiki Beta
blog.scraperwiki.com
on 05/10/2013
Excerpt: We’ve been teasing and tempting you with blog posts about the first few tools on the new ScraperWiki Beta for a while now. It’s time to let you try them out first-hand. As of right now, the new ScraperWiki Beta is open for you, your aunt, anyone, to sign up for a free community account: Check out beta. scraperwiki. com. We’re really excited. Not only does this mean all of our Classic Premium Account holders, and our new private beta applicants, have been settled into the new platform, but now regular Classic users get to try the new ScraperWiki out, for free. The new ScraperWiki beta is a little rough around the edges, but it can already do everything ScraperWiki Classic did, and more. As we (and you!... read the full post.
Tags: API-Evangelist, API-Stack, Harvesting, Scraping, Story
ScraperWiki lets anyone scrape Twitter data without coding
gigaom.com
on 05/10/2013
Excerpt: Summary: A new beta version of ScraperWiki makes it easy to relatively easy to scrape Twitter for certain phrases and get to work analyzing the data. It’s just one more way that data analysis is getting democratized. The Obama administration’s open data mandate announced on Thursday was made all the better by the unveiling of the new ScraperWiki service on Friday. If you’re not familiar with ScraperWiki, it’s a web-scraping service that has been around for a while but has primarily focused on users with some coding chops or data journalists willing to pay to have someone scrape data sets for them.... read the full post.
Tags: API-Evangelist, API-Stack, Harvesting, Scraping, Twitter
Mixing Up Great Data Without the Hassle of Scraping
feeds.developer.yahoo.net
on 05/06/2013
Excerpt: There is an awful lot of great data online which has the potential to power awesome next-generation apps and services. Open data, government data services, and many APIs provide relatively easy access to data which can be integrated with sufficient time and effort. However, the data your app or service needs is not necessarily available in a useful format. In fact, the vast majority of online data is locked up in web pages with no provision of data dumps or APIs in order to programmatically access it. Frequently, the solution to this problem has been the development of so-called "scrapers". While scrapers may initially solve the problem of data retrieval, they come with a number of issues.... read the full post.
Tags: API-Evangelist, API-Stack, Harvesting, Import.io, Scraping
Two ways you can help guide ScraperWiki’s new platform.
blog.scraperwiki.com
on 04/22/2013
Excerpt: You will have noticed some activity over the past few weeks, as we have begun reaching out about the new ScraperWiki platform. We’ve blogged about some of the new features, and have invited the first ever users outside the office to have a poke around the beta. That initial feedback has been immeasurably helpful, and has lead to bug-fixes, feature requests, and some directional suggestions which we can’t thank you enough for. But we need more. So, there are now two ways you can join the testing community, and – with absolutely no exaggeration – play a vital part in the future functionality, design, and direction of the new ScraperWiki. First, you can become a premium user of the New ScraperWiki.... read the full post.
Tags: API-Evangelist, API-Stack, Feedback, Harvesting, Scraping
Tools of the trade
blog.scraperwiki.com
on 03/14/2013
Excerpt: With the experience of a whole week of ScraperWiki, I am starting to appreciate the core tools of the professional Data Scientist. In the past I’ve written scrapers in Matlab, C# and Python. However, the house language for scraping at ScraperWiki is Python. It’s a good choice: a mature but modern language with a wide range of useful, easy to install libraries. As an interpreted language, it is easy to use in a very fast development cycle. There are a number of libraries which are core to the scraping process:
Requests is an HTTP library for reading web pages. It has a rather cleaner syntax than the standard urllib2 and has useful plugins such as requests_cache which gives you dead simple web page caching.... read the full post.
Tags: API-Evangelist, API-Stack, Data, Harvesting, Scraping, Tools
A small matter of programming
blog.scraperwiki.com
on 12/31/2012
Excerpt: For three years, we’ve been helping people get, clean and analyse data on the web. Our key insight was that you need to write code to do that, and we should make writing that code as easy as possible. Earlier this year, we realised that that isn’t enough. ScraperWiki Classic, as we now call the original ScraperWiki, falls into a gap. It’s neither flexible enough for most programmers, nor simple enough for most non-programmers. There’s a bunch of people who love it, right in the middle. Our plan was to push that sideways – add more features to make it useful to programmers, and add tools to make it useful to anyone. We listened and watched our users and found it wasn’t going to work[1].... read the full post.
Tags: API-Evangelist, Data, Harvesting, Scraping
The next evolution of ScraperWiki
blog.scraperwiki.com
on 12/21/2012
Excerpt: Quietly, over the last few months, we’ve been rebuilding both the backend and the frontend of ScraperWiki. The new ScraperWiki has been built from the ground up to be more powerful for data scientists, and easier to use for everyone else. At its core, it’s about empowering people to take a hold of their data, to analyse it, combine it, and make value from it. We can’t wait to let you try it in January. In the meantime, however, we’re pleased to announce that all of our corporate customers are already migrating to the new ScraperWiki for scraping, storing and visualising their private datasets. If you want data scraping, cleaning or analysing, then you can join them. Please get in touch.... read the full post.
Tags: API-Evangelist, Data, Harvesting, Scraping
Excerpt: Summary: A new beta version of ScraperWiki makes it easy to relatively easy to scrape Twitter for certain phrases and get to work analyzing the data. It’s just one more way that data analysis is getting democratized. The Obama administration’s open data mandate announced on Thursday was made all the better by the unveiling of the new ScraperWiki service on Friday. If you’re not familiar with ScraperWiki, it’s a web-scraping service that has been around for a while but has primarily focused on users with some coding chops or data journalists willing to pay to have someone scrape data sets for them.... read the full post.
Tags: API-Evangelist, API-Stack, Harvesting, Scraping, Twitter
Mixing Up Great Data Without the Hassle of Scraping
feeds.developer.yahoo.net
on 05/06/2013
Excerpt: There is an awful lot of great data online which has the potential to power awesome next-generation apps and services. Open data, government data services, and many APIs provide relatively easy access to data which can be integrated with sufficient time and effort. However, the data your app or service needs is not necessarily available in a useful format. In fact, the vast majority of online data is locked up in web pages with no provision of data dumps or APIs in order to programmatically access it. Frequently, the solution to this problem has been the development of so-called "scrapers". While scrapers may initially solve the problem of data retrieval, they come with a number of issues.... read the full post.
Tags: API-Evangelist, API-Stack, Harvesting, Import.io, Scraping
Two ways you can help guide ScraperWiki’s new platform.
blog.scraperwiki.com
on 04/22/2013
Excerpt: You will have noticed some activity over the past few weeks, as we have begun reaching out about the new ScraperWiki platform. We’ve blogged about some of the new features, and have invited the first ever users outside the office to have a poke around the beta. That initial feedback has been immeasurably helpful, and has lead to bug-fixes, feature requests, and some directional suggestions which we can’t thank you enough for. But we need more. So, there are now two ways you can join the testing community, and – with absolutely no exaggeration – play a vital part in the future functionality, design, and direction of the new ScraperWiki. First, you can become a premium user of the New ScraperWiki.... read the full post.
Tags: API-Evangelist, API-Stack, Feedback, Harvesting, Scraping
Tools of the trade
blog.scraperwiki.com
on 03/14/2013
Excerpt: With the experience of a whole week of ScraperWiki, I am starting to appreciate the core tools of the professional Data Scientist. In the past I’ve written scrapers in Matlab, C# and Python. However, the house language for scraping at ScraperWiki is Python. It’s a good choice: a mature but modern language with a wide range of useful, easy to install libraries. As an interpreted language, it is easy to use in a very fast development cycle. There are a number of libraries which are core to the scraping process:
Requests is an HTTP library for reading web pages. It has a rather cleaner syntax than the standard urllib2 and has useful plugins such as requests_cache which gives you dead simple web page caching.... read the full post.
Tags: API-Evangelist, API-Stack, Data, Harvesting, Scraping, Tools
A small matter of programming
blog.scraperwiki.com
on 12/31/2012
Excerpt: For three years, we’ve been helping people get, clean and analyse data on the web. Our key insight was that you need to write code to do that, and we should make writing that code as easy as possible. Earlier this year, we realised that that isn’t enough. ScraperWiki Classic, as we now call the original ScraperWiki, falls into a gap. It’s neither flexible enough for most programmers, nor simple enough for most non-programmers. There’s a bunch of people who love it, right in the middle. Our plan was to push that sideways – add more features to make it useful to programmers, and add tools to make it useful to anyone. We listened and watched our users and found it wasn’t going to work[1].... read the full post.
Tags: API-Evangelist, Data, Harvesting, Scraping
The next evolution of ScraperWiki
blog.scraperwiki.com
on 12/21/2012
Excerpt: Quietly, over the last few months, we’ve been rebuilding both the backend and the frontend of ScraperWiki. The new ScraperWiki has been built from the ground up to be more powerful for data scientists, and easier to use for everyone else. At its core, it’s about empowering people to take a hold of their data, to analyse it, combine it, and make value from it. We can’t wait to let you try it in January. In the meantime, however, we’re pleased to announce that all of our corporate customers are already migrating to the new ScraperWiki for scraping, storing and visualising their private datasets. If you want data scraping, cleaning or analysing, then you can join them. Please get in touch.... read the full post.
Tags: API-Evangelist, Data, Harvesting, Scraping
Excerpt: You will have noticed some activity over the past few weeks, as we have begun reaching out about the new ScraperWiki platform. We’ve blogged about some of the new features, and have invited the first ever users outside the office to have a poke around the beta. That initial feedback has been immeasurably helpful, and has lead to bug-fixes, feature requests, and some directional suggestions which we can’t thank you enough for. But we need more. So, there are now two ways you can join the testing community, and – with absolutely no exaggeration – play a vital part in the future functionality, design, and direction of the new ScraperWiki. First, you can become a premium user of the New ScraperWiki.... read the full post.
Tags: API-Evangelist, API-Stack, Feedback, Harvesting, Scraping
Tools of the trade
blog.scraperwiki.com
on 03/14/2013
Excerpt: With the experience of a whole week of ScraperWiki, I am starting to appreciate the core tools of the professional Data Scientist. In the past I’ve written scrapers in Matlab, C# and Python. However, the house language for scraping at ScraperWiki is Python. It’s a good choice: a mature but modern language with a wide range of useful, easy to install libraries. As an interpreted language, it is easy to use in a very fast development cycle. There are a number of libraries which are core to the scraping process:
Requests is an HTTP library for reading web pages. It has a rather cleaner syntax than the standard urllib2 and has useful plugins such as requests_cache which gives you dead simple web page caching.... read the full post.
Tags: API-Evangelist, API-Stack, Data, Harvesting, Scraping, Tools
A small matter of programming
blog.scraperwiki.com
on 12/31/2012
Excerpt: For three years, we’ve been helping people get, clean and analyse data on the web. Our key insight was that you need to write code to do that, and we should make writing that code as easy as possible. Earlier this year, we realised that that isn’t enough. ScraperWiki Classic, as we now call the original ScraperWiki, falls into a gap. It’s neither flexible enough for most programmers, nor simple enough for most non-programmers. There’s a bunch of people who love it, right in the middle. Our plan was to push that sideways – add more features to make it useful to programmers, and add tools to make it useful to anyone. We listened and watched our users and found it wasn’t going to work[1].... read the full post.
Tags: API-Evangelist, Data, Harvesting, Scraping
The next evolution of ScraperWiki
blog.scraperwiki.com
on 12/21/2012
Excerpt: Quietly, over the last few months, we’ve been rebuilding both the backend and the frontend of ScraperWiki. The new ScraperWiki has been built from the ground up to be more powerful for data scientists, and easier to use for everyone else. At its core, it’s about empowering people to take a hold of their data, to analyse it, combine it, and make value from it. We can’t wait to let you try it in January. In the meantime, however, we’re pleased to announce that all of our corporate customers are already migrating to the new ScraperWiki for scraping, storing and visualising their private datasets. If you want data scraping, cleaning or analysing, then you can join them. Please get in touch.... read the full post.
Tags: API-Evangelist, Data, Harvesting, Scraping
Excerpt: For three years, we’ve been helping people get, clean and analyse data on the web. Our key insight was that you need to write code to do that, and we should make writing that code as easy as possible. Earlier this year, we realised that that isn’t enough. ScraperWiki Classic, as we now call the original ScraperWiki, falls into a gap. It’s neither flexible enough for most programmers, nor simple enough for most non-programmers. There’s a bunch of people who love it, right in the middle. Our plan was to push that sideways – add more features to make it useful to programmers, and add tools to make it useful to anyone. We listened and watched our users and found it wasn’t going to work[1].... read the full post.
Tags: API-Evangelist, Data, Harvesting, Scraping
