Case Study: Data Analysis with Python and Java - Step by-Step Guide

Data analysis is a crucial process for any business that wants to make informed decisions based on their data. In this case study, we will walk through the steps of performing data analysis for a retail company using Python and Java.

 

1. Data Acquisition and Preparation:

To begin with, we need to collect relevant sales data from various sources, such as databases, web, or APIs. We will then clean and prepare the data for analysis by removing duplicate entries and handling missing values. In this case, our retail company collects sales data from their database and prepares it by converting it into a CSV format.

 

import pandas as pd 
import numpy as np 

#reading data from database
sales_data = pd.read_sql_query('SELECT * FROM Sales', con=database_connection)

#removing duplicate entries
sales_data.drop_duplicates(inplace=True)

#handling missing values
sales_data.dropna(inplace=True)

#converting to CSV format 
sales_data.to_csv('sales_data.csv', index=False)

Java code for Data Acquisition and Preparation involves using tools like Apache Hadoop ecosystem, HDFS, and Hive to store, retrieve, and analyze data from various sources.

//using HDFS to store and retrieve data
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);
Path outputPath = new Path("/user/output");
fs.get(outputPath,outputPath + "output.csv");

//using Hive to query and analyze data 
import org.apache.hadoop.hive.ql.exec.spark.session.SparkSession;
import org.apache.hadoop.hive.ql.exec.spark.SparkTask;
SparkSession spark = SparkSession.builder().getOrCreate();
spark.sql("SELECT * FROM sales_data ORDER BY Sales DESC LIMIT 10");

 

2. Understanding Your Data:

Once the data is cleaned and prepared, we need to gain a deeper understanding of it. We can do this by visualizing the data using plots, histograms, and summary statistics. This step will help us identify any patterns, trends, or relationships in the data. Let's use the seaborn library in Python to visualize our sales data.

 

import seaborn as sns 

#visualizing sales trends 
sns.lineplot(x='Year', y='Sales', data=sales_data)

#visualizing sales by location 
sns.barplot(x='Location', y='Sales', data=sales_data)

#calculating summary statistics 
sales_data[['Sales', 'Quantity', 'Price']].describe()

//using Java to visualize sales data 
import org.jfree.chart.ChartFactory; 
import org.jfree.chart.ChartPanel; 
import org.jfree.chart.JFreeChart; 
import org.jfree.chart.plot.PlotOrientation;
import org.jfree.data.category.DefaultCategoryDataset;

//create dataset 
DefaultCategoryDataset dataset = new DefaultCategoryDataset();
//add data to dataset 
dataset.addValue(10000, "Sales", "2018");
dataset.addValue(15000, "Sales", "2019");
dataset.addValue(20000, "Sales", "2020");

//create chart 
JFreeChart chart = ChartFactory.createBarChart("Sales Data", "Year", "Sales", dataset, PlotOrientation.VERTICAL, false, true, false);

//create chart panel 
ChartPanel chartPanel = new ChartPanel(chart);
//add chart panel to frame 
jFrame.add(chartPanel);
//set frame size and make it visible 
jFrame.setSize(500, 400); 
jFrame.setVisible(true);

 

3. Perform Data Analysis using Python:

Python has a rich set of libraries for data analysis, such as pandas, numpy, and scikit-learn. These libraries provide various functions and methods for data manipulation, visualization, and statistical analysis. We can use these tools to gain insights into our data and make informed decisions.

 

#grouping sales by product category 
sales_data.groupby('Category')['Sales'].sum().sort_values(ascending=False)

#grouping sales by location and time period 
sales_data.groupby(['Location', 'Year'])['Sales'].sum()

4. Perform Data Analysis using Java:

Java is another popular programming language for data analysis. The Apache Hadoop ecosystem offers a suite of tools for data analysis in Java, including HDFS, MapReduce, and Hive. These tools enable us to work with large datasets and perform distributed processing for faster analysis.

//using HDFS to store and retrieve data 
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);
Path outputPath = new Path("/user/output");
fs.get(outputPath,outputPath + "output.csv");

//using Hive to query and analyze data 
import org.apache.hadoop.hive.ql.exec.spark.session.SparkSession;
import org.apache.hadoop.hive.ql.exec.spark.SparkTask;
SparkSession spark = SparkSession.builder().getOrCreate();
spark.sql("SELECT * FROM sales_data ORDER BY Sales DESC LIMIT 10");

 

5. Building Predictive Models:

Once the data has been analyzed, we can use machine learning algorithms to build predictive models. In both Python and Java, there are libraries and frameworks available for creating predictive models, such as scikit-learn, TensorFlow, and Weka. These tools help us build models that can make predictions and identify patterns in new data.

 

#using scikit-learn to build a linear regression model 
from sklearn.linear_model import LinearRegression 
X = sales_data[['Sales', 'Quantity', 'Price']]
y = sales_data['Profit']
reg_model = LinearRegression()
reg_model.fit(X, y)

//using Weka to build a decision tree model 
import weka.core.Instances;
import weka.classifiers.trees.J48;
import weka.classifiers.Evaluation;

//loading data from CSV file
Instances data = new Instances(new BufferedReader(new FileReader("sales_data.csv")));
data.setClassIndex(data.numAttributes() - 1);

//building the decision tree model
J48 decision_tree = new J48();
decision_tree.buildClassifier(data);

//evaluating the model
Evaluation eval = new Evaluation(data);
eval.crossValidateModel(decision_tree, data, 10, new Random(1));
System.out.println(eval.toSummaryString());

 

6. Evaluate and Validate Models:

After building predictive models, it is essential to evaluate and validate their performance. This involves testing the model on unseen data and measuring its accuracy, precision, and recall. We can use techniques like cross-validation and confusion matrices to evaluate our models.

 

#validating model using cross-validation 
from sklearn.model_selection import cross_val_score 
scores = cross_val_score(reg_model, X, y, cv=10)
print("Mean accuracy: %0.2f" % (scores.mean()))

#evaluating model using confusion matrix 
from sklearn.metrics import confusion_matrix 
y_pred = reg_model.predict(X_test)
conf_mat = confusion_matrix(y_test, y_pred)
print('Confusion Matrix:\n', conf_mat)

//using Weka library for cross-validation
import weka.classifiers.Evaluation;
import weka.classifiers.functions.Logistic;
import weka.core.Instances;
import java.io.*;
BufferedReader breader = new BufferedReader(new FileReader("sales_data.arff"));
Instances instances = new Instances(breader);
instances.setClassIndex(instances.numAttributes() - 1);
Logistic logistic = new Logistic();
Evaluation eval = new Evaluation(instances);
eval.crossValidateModel(logistic, instances, 10, new Random(1));
System.out.println("Mean accuracy: " + eval.pctCorrect());

 

7. Communicate Results:

The final step in data analysis is to communicate the results to stakeholders. We can do this through visualizations, reports, and presentations. Python and Java provide various tools for creating visualizations and presenting the results of our data analysis, making it easier for stakeholders to understand and make informed decisions.

 

#creating a visualization to present sales data trends 
sns.lineplot(x='Year', y='Sales', hue='Location', data=sales_data)

#creating a visualization to present top-selling products 
sns.barplot(x='Product', y='Sales', data=sales_data)

#creating a report to present the performance of our predictive model
print("Model Accuracy: %0.2f%%" % (reg_model.score(X_test, y_test) * 100))

/creating a line chart to visualize sales trends 
import com.itextpdf.text.*;
import com.itextpdf.text.pdf.*;

//creating a new document
Document document = new Document();
PdfWriter.getInstance(document, new FileOutputStream("sales_report.pdf"));
document.open();

//adding a title to the report
Paragraph title = new Paragraph("Sales Report", FontFactory.getFont(FontFactory.HELVETICA, 18, Font.BOLD));
document.add(title);

//creating and adding the line chart
JFreeChart lineChart = ChartFactory.createLineChart("Sales Trends", "Year", "Sales", dataset, PlotOrientation.VERTICAL, true, true, false);
ChartUtilities.writeChartAsPNG(contentStream, 0, 0, 500, 400, lineChart, 300);
document.add(new Paragraph("Sales Trends by Year:"));
document.add(new Paragraph(new Chunk(lineChart, 0, 0)));

//adding a table to show top-selling products 
PdfPTable table = new PdfPTable(2);
table.addCell("Product");
table.addCell("Sales");
//adding data to table
for(int i=0; i

 

Conclusion:

In summary, by following these steps, we were able to perform data analysis for our retail company using Python and Java. We collected and prepared our data, gained insights into it, built predictive models, evaluated their performance, and communicated the results to stakeholders. This process helps businesses make informed decisions based on their data, leading to more successful strategies and outcomes.

MyExamCloud Study Plans

Java Certifications Practice Tests - MyExamCloud Study Plans

Python Certifications Practice Tests - MyExamCloud Study Plans

AWS Certification Practice Tests - MyExamCloud Study Plans

Google Cloud Certification Practice Tests - MyExamCloud Study Plans

Aptitude Practice Tests - MyExamCloud Study Plan