https://github.com/devasur/kivasearch
Currently Hosted @ Kiva Search on Amazon Cloud
Just finished writing my full blown JavaScript search tool for Kiva and I was too excited to keep it a secret anymore 
. I have been an involved kiva lender for many years now. For the last couple of years I have been using the nifty tool kivalens to search and locate suitable loans to lend to. It has worked great for me – extremely satisfied. The only challenge had been that it is Silverlight based. So does not work easily on Linux machines. I took kivalens’s functionality as a base to learn NodeJS by converting it into a pure HTML5 single page client with a NodeJS backend. It currently searches using almost all the filtering options the mother application has. It still lacks some features but those are not really important to me. But if time permits I would love to make it full fledged and publish open sourced.
The application works like this. It uses the public kiva RESTful APIs to get newest loans from Kiva in JSON format. To it it adds in Field Partner information. Some of the field partner information is not available through public APIs. So it uses server side webpage scrapping to get this additional information. It saves it all into a database. Couple of background jobs keeps this information updated. These jobs also purged fully funded loan information from the database so that its size is in check.
The client UI is fairly simple. It has some draggable sliders to mark min-max for some key lending parameters. When the parameters are changed matching loans are fetched and rendered as a list. User can click through to complete lending. It is simple and it works.
The serve side is NodeJS. I have used ‘express‘ as a web framework to manage the listening thread. MongoJS is used to connect and manage data stored in MongoDB. Three setTimeout() background jobs synchronizes Kiva datasets with local database. For webpage scrapping to collect additional partner information for more search options I used node-htmlparser and node-soupselect two brilliant node modules.
For the client side I went with Twitter Bootstrap for basic scaffolding. JQuery is used for managing various AJAX calls. Handlebarjs is used as the templating framework for rendering loan data.
One great side benefit of this learning was that I got introduced to a super editor called Sublime Text. It supports a huge number of programming languages and has a very flexible extension system to enhance its features. Writing NodeJS JavaScript on this IDE was like silk. Very impressed.
For a JavaScript expert this would be child’s play. But for me, someone who 90% time writes no-ui server code in Java, this whole project was a small scale Mt. Everest. I learnt so much of JavaScript and about the amazing collaborative development that is happening on the NodeJS front, The simplicity of NodeJS and the completeness of its support system in terms of all necessary tools and libraries is just amazing. I will continue to learn NodeJS by improving on what I have now.
An ambitious product roadmap for the next 30 days.
- Ability to save and retrieve search criteria based on some user credentials.
- A notification system that can push loan availability information to user from the backend even if user is not using the client.
- Repackage the client JS as a PhoneGap project for iOS and Android.
- More search criteria options.
- More visualization options for the loan and portfolio demographics.
Exciting times ahead!
Recent Comments