Inspiration
In our day-to-day lives, public officials and representatives can be of great help in making sure the citizens enjoy the necessary rights, safety, and public infrastructure. It is always a hassle to find the important public contacts in a single place since the information is scattered in multiple places/sites In India, the problem is even higher when the digitization of public information has just started getting streamlined
What it does
The information about the public representatives and district administrators is gathered from multiple portals by performing web scrapping using UI Path RPA scripts. The data are stored in Quickbase (low-code web application platform) tables, and a dashboard is created with multiple search options
How we built it
UI Path Studio is used to automate the UI navigation steps to scrap the data from the identified public-facing portals, pre-process them and insert them in Quickbase using HTTP web activity. A generic integration sequence file is created to make JSON-based RESTful API calls to insert the data in Quickbase. A dashboard is created in Quickbase with a global search option for Contacts data and specific search on parliament and assembly members data
Challenges we ran into
1) To create a generic sequence for making multiple HTTP API calls to Quickbase with different fields of data that are scraped from different kinds of web portal 2) The web portals are not designed consistently and scaping the data for different entities and states were a big challenge with different scenarios that were to be handled while creating the workflow 3) Finding out the URLs for different states and districts and automating them to capture the URLs dynamically to make them run seamlessly was one of the key challenges
Accomplishments that we're proud of
Generic sequence created with another platform that is reusable The robust UI automation scripts were created for scrapping the different sub sites created for each site and for each district
What we learned
Nuances of working with web elements in UI Path HTTP web requests in UI Path Managing multiple scripts linked to each other To design the sequence that is a robust and easy-flowing UI automation script
What's next for Public Contacts Directory Portal
1) To add more entities like gathering the village level public contacts 2) To make use of Quickbase to configure the variations in different UI automation to come up with more generic UI path scripts
Log in or sign up for Devpost to join the conversation.