Vivek Kumar

React Axios Cheerio Parsing GitHub Responses

Vivek Kumar

Nov 26, 2019
7 Min read
55 Views

Nov 26, 2019
7 Min read
55 Views

Web Development

React

Introduction

The service dimension of the React community is vast, and there are numerous features that you can access with React that you cannot avail in other transition frameworks. Among the service components you can access in React, Axios Cheerio is the one that can help you retrieve critical data from a server when required or for convenience.

Axios is a promise-centered HTTP client meant for a browser as well as for node.js. As procurement of data is one of the key functions of Axios, we generally use it to get HTML via any specifically chosen website. Axios Cheerio carries features and functions like jQuery, but it is specifically meant for use with a server. Cheerio is currently one of the most trusted ways to pick selective or the whole content visible in Axios results. It is worth mentioning that fs is a type of node module that we can apply in order to write the fetched content into the JSON file.

This guide will teach you to use React Axios and Cheerio parsing in order to retrieve data from GitHub. As most of us usually depend on the React component framework to execute animation transitions with the range of library props accessible through it, its role in the procurement of Git data is significant.

The retrieval of data from the hosting service provider is crucial for scheduled deadlines and working on various projects simultaneously. GitHub is one of the most popular hosting providers for software development version control and uses Git, a distributed version-control system meant for tracking down changes in source code during different phases of software development.

As we polish our knowledge through this guide, we will work through a common challenge to making requests from Axios to retrieve data from GitHub and learn the pertinent countermeasures to this challenge.

Data Retrieval Failure

In order to demonstrate the standard commands associated with data retrieval using Axios, let’s consider an example. Suppose your request from Axios to GitHub has been met with failure error. We’ll assume that all necessary measures have been taken from the client’s side, but there is a certain glitch resulting in the failure and so far, it is unidentifiable.

This is the standard input command for the data retrieval:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
componentDidMount() {    
    axios
      .get('https://github.com/users/lukeschlangen/contributions',{
        headers: {
          'Access-Control-Allow-Origin': '*'
        }
      })
      .then(res => {
        this.streakCounter(res);
      })
      .catch(err => console.log(err));
  }

  streakCounter(body) {
    const $ = cheerio.load(body);
    var data = [];
    $('svg').find('rect').each(function(index, element) {
        data.push({
            count: parseInt($(element).attr('data-count')),
            date: new Date($(element).attr('data-date'))
        })
    });

    var yesterday = new Date();
    yesterday.setDate(yesterday.getDate() - 1);

    data = data.sort(function(a, b) {
        return new Date(b.date) - new Date(a.date);
    }).filter(function(el) {
        return el.date.getTime() <= yesterday.getTime();
    });
    var streakCount = 0;
    for (var i = 0; i < data.length; i++) {
        if (data[i].count == 0) {
            break;
        }
        streakCount++
    }

    console.log('streakCount:', streakCount);

  }

jsx

This example demonstrates a frequent error in response. The common error (error I) notification which is displayed on the server in response to the failed data retrieval request will look something like the following:

Failed to load https://github.com/users/lukeschlangen/contributions: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:3000' is therefore not allowed access. The response had HTTP status code 404.

Simple Solutions

One of the key reasons React Axios Cheerio fails to acquire and interpret the GitHub response is that, for successful transmission and response to the data retrieval request, there are certain server-side changes required of GitHub. It is required to be authorized from the server end of the company. Please note that most of the hosting service plans offered by GitHub generally include the authorization facility, but it is required to be enabled by a special request from the client’s end.

Apart from getting approval from the GitHub server, there is an alternative way to get the GitHub response. If you are using a CORS (Cross-Origin Resource Sharing) proxy server, it will be a lot easier to put forward the information and get access to the data. You can even run the proxy server in your secure private network.

It is not essential to use a third party CORS proxy server in order to retrieve the data, but setting up your own personal proxy server is also not a daunting task. You can deploy a proxy server on the popular platform Heroku with few commands in a matter of minutes. A glimpse of how to set up a proxy server using Heroku is shown below:

1
2
3
4
5
git clone https://github.com/Rob--W/cors-anywhere.git
cd cors-anywhere/
npm install
heroku create
git push heroku master

bash

Conclusion

The process of parsing GitHub responses via Axios is much easier than most people think. As we learned in the guide above, a problem might appear in the form of different error codes, but they can easily be overcome with a simple communication passage authorization from GitHub or by creating a CORS proxy server on a reliable platform within a matter of minutes. And of course, services will remain transparent and more secure than ever.