Sanket Kumar Gautam SKG
About     Archive     Feed    

Apparate

Apparate is a command line tool to synchronize HackerRank Submissions with GitHub. I thought about this idea when I wanted to maintain a balance between Competitive Programming & Open Source Projects/Contributions. I just wanted some way to aggregate my online presence at one place. I searched a lot about any such existing service or tools for the same, but I couldn’t find anything.

Then, I thought to give it a try and build one for myself. It’s just my personal choice to start with HackerRank. Anyways, it can be later extended to support other Online Judges & Platforms.

Here are few tasks which Apparate does:

  • It creates a Submissions Repository on GitHub
  • It fetches new ‘Accepted’ Submissions from HackerRank
  • It performs code formatting (currently for C++ only)
  • It pushes new Submissions to Submissions Repository
  • Scheduled with ‘crontab’ it can synchronize the Repo regularly

Usage

It can be used by simply typing apparate on terminal as:

Usage: apparate [OPTIONS]

  Tool to Synchronize HackerRank Submissions with GitHub

Options:
  --repo TEXT    Name of GitHub repository to store submissions
  --user TEXT    Username of your HackerRank account
  --passwd TEXT  Login Password of your HackerRank account
  --token TEXT   GitHub Access Token with repository privileges
  --help         Show this message and exit.

Using apparate with crontab will automatically update the submissions repository regularly and makes our task much easier. It’s recommended to deploy it on some cloud instance or AWS Lambda (for better savings).

Tech Stack Used

Apparate is developed with Python(version 3.6.5) as it’s good for rapid development & also supports Selenium.

I used following python libraries for development:

  • Setuptools: Setuptools is a fully-featured, actively-maintained, and stable library designed to facilitate packaging Python projects. It’s used to make apparate easily installable just by using pip.
  • Selenium (v3.12.0): Selenium is a portable software testing framework for web applications. It’s used for fetching new submissions from HackerRank.
  • PyGitHub (v1.39): GitPython is a python library used to interact with git repositories. It’s basically a wrapper for GitHub API. It’s used to create & maintain user’s submissions repository.
  • Click (v6.7): Click is a Python package for creating beautiful command line interfaces.

Apart from those, I also used Mozilla Firefox v59.0.2 and Geckodriver v0.20.1 with Selenium. It uses official GitHub REST API v3.

Project Structure

Here’s the project structure for Apparate:

├── README.md
├── requirements.txt
├── scripts
│   ├── apparate.py
│   ├── config.py
│   ├── __init__.py
│   └── spider.py
└── setup.py

README.md contains installation & usage Instructions for Apparate Tool.

Here, setup.py is created with setuptools for packaging apparate, which can be easily installed by executing (inside Apparate/ directory) -

                pip install --editable .

scripts/ directory contains the main code files:

  • spider.py - contains code for fetch new submission, pretify code etc.
  • apparate.py - contains code for check updates, update repo etc.
  • config.py - declares some configuration variables

Example

Below is the output for example run of Apparate:

Executing Apparate on Mon, 14 May 2018, 11:06:00
GitHub Authentication Successful
Submissions Repo already exists
submissions size 0
Headless Firefox Initialized
HackerRank Login Successful
pagination params, 1 - 17
95 new submission(s) found.
Fetching code for new submissions..
- fetching code for submission 1. Diagonal Difference
- fetching code for submission 2. Birthday Cake Candles
- fetching code for submission 3. Make It Anagram
- fetching code for submission 4. Employee Names
- fetching code for submission 5. Weather Observation Station 8
- fetching code for submission 6. Points On a Line
   .
   .
   .
- updating repo with submission 91 
   -- created new file 
- updating repo with submission 92 
   -- created new file 
- updating repo with submission 93
   -- created new file 
- updating repo with submission 94 
   -- created new file 
- updating repo with submission 95 
   -- created new file 
submissions.txt updated successfully 
time taken to Apparate is 4 min(s), 17 sec(s)

How it works?

Apparate works in the following way:

  • Firstly, it checks whether the submissions repo is already present (If not, then creates one).
  • Checks if submissions.txt (useful for keeping track of submissions in the repo) is already present (if not, then create it with empty submissions array).
  • It then performs login to the user’s HackerRank account
  • It fetches the list of all ‘accepted’ submissions which are not in GitHub repository.
  • Then, it fetches code for all new submissions and format code (only for C++).
  • And then, it creates/updates a file in GitHub Repo for each new submission and commits it.
  • In the end, it updates the submissions.txt in the GitHub Repo by adding the new list of submissions.

Conclusion & Further Plans

Apparate turned out to be a quite helpful tool to me and to other fellow developers. I just made it for myself, but later generalized it for everyone. It gives immense pleasures to see it helping others. I’ll be actively involved with Apparate by fixing bugs, improving code and maintenance.

Also, I would like to inform that Apparte is open to contributions and adding support for new platforms such as SPOJ, Codechef, Codeforces etc.