Building Gaffa - a web automation and scraping API service

James Mundy  • Peerlist

Getting reliable web data at scale is tough and getting harder all the time. Gaffa provides an API that makes it simple again. Gaffa will take care of all the complexity like residential proxies, scaling, infrastructure, failures and data parsing so you can concentrate on your business and not fixing broken scraping pipelines. Using Gaffa's REST API you can send a single request to perform a huge number of actions like scrolling and clicking mimicking natural human browsing patterns as well as extracting page data as images, markdown, html or, using LLM, JSON. Because Gaffa is designed to work even against the hardest to scrape sites it also has a stealth mode which uses real browsers, solves CAPTCHAs and uses residential proxies to ensure you get the data you need with minimum stress.

Gaffa

- Foundbite back-end running the apps and website. - Services used include: Azure Cloud Service, Azure App Service, Cloudant CouchDB, Azure Storage (Blob, Queue), Redis Cache, Azure SQL, WebJobs and Notification Hubs.

Foundbite API

Average rating of 4.4/5 from over 200 reviews on the Google Play Store. Promoted on all Sony devices thanks to a partnership using the Sony Camera. Worked alongside Alfie developing app using Xamarin, primarily focused on shared code in Portable Class Libraries.

Foundbite for Android

Built using Xamarin using Portable Class Libraries developed for other Foundbite apps. Recently released but solid feedback so far. Designed and developed the app using Xamarin Studio and Interface Builder (UI), C# and Xamarin for the rest. Utilised and bound many third-party libraries to use within the Xamarin environment.

Foundbite for iPhone

Exclusive to Windows Phone initially thanks to funding from Microsoft and Nokia through the AppCampus program. Received 600+ reviews with average rating of 4.5/5 and used all over the world. Worked on all aspects of the application including UI design (XAML), localisation, marketing and  development. Implemented business logic in portable class libraries to share 60% of code between platforms other platforms using Xamarin and MVVM design pattern.

Foundbite for Windows Phone

- ASP.NET MVC application - Highlights best foundbites from around the world and allows users to share their foundbites via Facebook and Twitter. - Allows deeplinking into appropriate app store for device or into Foundbite app.

Foundbite Website

- Travel alarm app that travellers up just before reaching their destination. - First app developed for Windows Phone. - 5000+ downloads, 4/5 average rating.

Travelnapp for Windows Phone

- Allowed user to take a picture and then guessed age using the Face.com API. - Featured on the Windows Store several times in various countries, downloaded over 11,000 times. - Offended quite a few people! - Built server-side back-end using Windows Azure (Table Storage, Block Blobs, Queue Messages and Cloud Services). - Developed and designed over the course of a summer. Years later Microsoft released http://how-old.net/ which went viral (and was remarkably similar).

VizAge for Windows Phone

- First live-tile based advent calendar for Windows Phone - 20% of the profits donated to the Railway Children charity. - Promoted by Microsoft.