Data Acquisition, Web Scraping, APIs - Python & MySQL
Project scope
Categories
Information technology Software developmentSkills
beautifulsoup web scraping application programming interface (api) python (programming language) selenium (software) xpath data acquisition scrapy (web crawler) hypertext markup language (html) data retrievalWe at Speedy Snacks would like to improve our back end data acquisition systems. Currently we have to manually copy transaction data from partner websites that is unavailable through an API so we would like to automate the process.
We are looking for students proficient in Python. Some understanding of Scrapy, Scrapy Splash, Selenium, BeautifulSoup, regex, xpath, javascript, CSS, and HTML would also go a long way. They will be tasked with automating the process of copying transaction data from our partners websites and storing it in our database for further processing.
To achieve the project goal student will need to:
- Examine how data is retrieved through partner web portals
- Design a process to automate data retrieval
- Develop a software application to automate data retrieval
I will be available for scheduled weekly meetings to evaluate how the project is going, provide input, and help with any technical challenges. I'll also be available via email, telephone, and most messaging apps to help with any challenges as they arise.
I'll demonstrate and explain our current systems and processes and assist the student with development process.
About the company
Speedy Snacks is expanding across Canada. We have developed technology to track advertising, drive sales, and deliver product to consumers helping convenience stores and other small businesses earn additional revenue. We create & deploy social media and ad campaigns to drive sales and partner with independent convenience store operators and third party food delivery apps to fill and deliver orders. We track ad spends, social media campaigns, and product sales to calculate the most profitable products to promote and ad campaigns to run.