Human Factors in Data Visualization

Data visualization plays a crucial role in exploratory data analysis and communicating business insights. However, there are several human factors that are also an important part of generating unbiased and useful results. In this guide, you will learn about these concepts, which will help you generate meaningful business insights through data visualization using R.

In this guide, we'll be using a fictitious dataset of loan applications containing 572 observations and 12 variables:

1. UID: Unique identifier for an applicant

2. Marital_status: Whether the applicant is married ("Yes") or not ("No")

3. Dependents: Number of dependents of the applicant

4. Is_graduate: Whether the applicant is a graduate ("Yes") or not ("No")

5. Income: Annual income of the applicant (in USD)

6. Loan_amount: Loan amount (in USD) for which the application was submitted

7. Term_months: Tenure of the loan

8. Credit_score: Whether the applicant's credit score is good ("Satisfactory") or not ("Not Satisfactory")

9. approval_status: Whether the loan application was approved ("1") or not ("0")

10. Age: The applicant's age in years

11. Sex: Whether the applicant is a male ("M") or a female ("F")

12. Purpose: Purpose of applying for the loan

Let's start by loading the required libraries and the data.

Output:

The output shows that the dataset has six numerical variables (labeled as 'int') and six categorical variables (labeled as 'chr'). We will convert these into factor variables, with the exception of the UID variable, using the line of code below.

Output:

Before starting with visualization, it is important to free the data from duplicates because they increase computation time and decrease model accuracy. In our dataset, UID is the unique identifier variable and will be used to drop the duplicate records. The first line of code below uses the duplicated() function to find the duplicates, while the second line prints the number of such records.

Output:

The output shows three duplicate records, which are dropped from the data using the first line of code below. The second line prints the new dimension of the data - 569 observations and 12 variables.

Output:

We are now ready to understand the data and the importance of human factors in visualization.

When you have a lot of variables to analyze, it becomes important to decide which one to visualize. The human ability to use subject matter expertise and domain knowledge to identify such features is required. A few examples are discussed in the section below.

In the loan application dataset, an assumption may be that age plays an important role in loan approvals. We'll use a histogram to visualize the distribution of the numerical variable Age. A histogram shows the number of data values within a bin for a numerical variable, with the bins dividing the values into equal segments. The vertical axis of the histogram shows the count of data values within each bin. The code below plots a histogram of the Age variable.

Output:

The above chart shows there are negative age values, which is impossible. Let's look at the summary of Age variable.

Output:

The output confirms that the minimum value of the variable, Age, is -12. This is incorrect and must be replaced, which brings us to an important human factor in data science: the ability to make assumptions when dealing with incorrect records. For loan applications, we'll assume that the minimum age should be 18 years. This means that we'll remove records of applicants below 18 years of age.

The first couple of lines of code below give the number of records in the dataset that have an age below 18 years. We have three such records. The third line of code removes such records, while the fourth line prints the dimensions of the new data: 566 observations and 12 variables.

Finally, look again at the summary of the age variable. This shows that the range of age is now between 23 to 76 years, indicating that the correction has been made.

Output:

Let's look at the plot again. The output shows that the distribution is much more reasonable.

Output:

It was easy to detect the incorrect entries in the age variable. However, at times it is important to identify anamolies or outliers in the data. Let's look at an example of the Income variable.

Output:

The above visualization shows that income levels are not normally distributed, and the distribution is right skewed. Confirm this with the summary() function, as shown below.

Output:

The summary shows that the minimum and maximum income levels are $17,320 and $844,490, respectively. This is a big range, indicating outliers. In order to better understand the distribution, use the quantile() function, which gives the first to hundredth percentile value of the variable.

Output:

Treat these outliers using the method of flooring and capping. The first line of code below floors the lower values at the fifth percentile value. Similarly, the second line caps the extreme values at the 95th percentile value. The third line prints the summary indicating that the correction has been done. Selecting the percentile value is a subjective decision after exploring the data.

Output:

One of the important visualization tasks is to find relationships between two or more variables. Depending on the data type, you can use visualization. The human factor that governs this choice is the ability to create a hypothesis and then examine it visually.

The first hypothesis we'll examine is whether or not there is a linear relationship between the variables Age and Income. We'll use a scatterplot, a popular technique to visualize the relationship between two quantitative variables. It can be plotted using the code below.

Output:

The output above shows that there is no linear relationship between the ages and income levels of the applicants. Similarly, we can test the hypothesis for other numerical variables as well.

The second hypothesis we'll examine is whether or not the income levels of the applicants have any bearing on the purpose of the loans. The visualization technique we'll use is the boxplot.

A boxplot is a standardized way of displaying the distribution of data based on a five-number summary (minimum, first quartile (Q1), median, third quartile (Q3), and maximum). The line of code below plots the distribution of the numeric variable Income against the categorical variable Purpose.

Output:

From the output above, we can deduce that the average income of the applicants does not vary significantly against the purpose levels. We also observe more outliers for applicants who applied for educational and personal loans.

One of the most crucial decisions that visualization experts face is the number of dimensions to be kept for visualization. In simple terms, dimensions refer to the number of variables being captured. For instance, in the examples above, we considered one and two dimensions. Increasing the number of variables can provide more information, but also increases the risk of making it difficult to interpret, something referred to as the curse of dimensionality.

The code below creates a 3-dimensional chart where the variables Income and Loan_amount are represented on the two axes, while the third dimension, approval_status, is added to the chart using the color argument.

Output:

The above chart is a good example of how to show more than two variables in a two-dimensional chart. However, the ggplot() library provides the facility to add more dimensions. In the above chart, you can add a fourth dimension, Dependents, using the size argument. This is done using the code below, where the size of the circle indicates the number of dependents.

Output:

In the previous chart, we represented four dimensions. You could still add more dimensions to the chart, but you'll run the risk of making it extremely difficult for a non-technical audience to interpret. To illustrate, use the GGally package in R, which uses the ggpairs() function to visualize pairwise relationships across variables.

The first line of code below creates a dataframe consisting of two continuous and three categorical variables, while the second line creates the plot.

Output:

The single chart above represents multiple dimensions, but it is very difficult to extract insights from it. Human judgment is required to decide on the dimensionality used in visualization.

In this guide, you have learned how human factors play a significant role in various elements of data visualization. You learned these concepts using the powerful ggplot() library in R. These techniques will help you in making decisions about how to visualize the data and extract meaningful business insights.

To learn more about data science with R, please refer to the following guides:

1. Interpreting Data Using Descriptive Statistics with R

2. Interpreting Data Using Statistical Models with R

3. Time Series Forecasting Using R

4. Hypothesis Testing - Interpreting Data with Statistical Models

5. Machine Learning with Text Data Using R

6. Visualization of Text Data Using Word Cloud in R

7. Exploring Data Visually with R

8. Coping with Missing, Invalid and Duplicate Data in R

9. Reshaping Data with R

10. Working with Data Types in R

11. Splitting and Combining Data with R

12. Linear, Lasso, and Ridge Regression with R

13. Binomial Coefficient Analysis with R

14. Implementing Marketing Analytics in R: Part 1

15. Implementing Marketing Analytics in R: Part 2