In this exercise we will build a Decision Tree Regression Model to find out key variables that impact credit card balances

This data is taken from “An Introduction to Statistical Learning with Applications in R” available at http://www-bcf.usc.edu/~gareth/ISL/index.html

• Import basic packages and interactive shell to print many statements on one line
• Import data set and check ‘head’ and ‘info’ of the data. Categorical variables are represented as “Object” in the below table and Numerical variables as “Int” or “Float
• Remove unnecessary columns using ‘iloc’ and print random sample of data using ‘sample’ method
• Create new variable in the data frame. This is the dependent variable
• Drop variables from dataframe using ‘.drop’ – ‘Cards’ and ‘Balance’ being dropped
• Explore data bit more about the dataframe- describe, info, shape, insert blank lines in print statements, count missing values, column names etc.
• Find out the mean and median values of the numerical variables by categorical variables by running a simple ‘for loop’
• Do plotting using Seaborn (sns) package- Seaborn Pairplot. Possible values for the color palette are huge and can be selected from -Possible values are: Accent, Accent_r, Blues, Blues_r, BrBG, BrBG_r, BuGn, BuGn_r, BuPu, BuPu_r, CMRmap, CMRmap_r, Dark2, Dark2_r, GnBu, GnBu_r, Greens, Greens_r, Greys, Greys_r, OrRd, OrRd_r, Oranges, Oranges_r, PRGn, PRGn_r, Paired, Paired_r, Pastel1, Pastel1_r, Pastel2, Pastel2_r, PiYG, PiYG_r, PuBu, PuBuGn, PuBuGn_r, PuBu_r, PuOr, PuOr_r, PuRd, PuRd_r, Purples, Purples_r, RdBu, RdBu_r, RdGy, RdGy_r, RdPu, RdPu_r, RdYlBu, RdYlBu_r, RdYlGn, RdYlGn_r, Reds, Reds_r, Set1, Set1_r, Set2, Set2_r, Set3, Set3_r, Spectral, Spectral_r, Vega10, Vega10_r, Vega20, Vega20_r, Vega20b, Vega20b_r, Vega20c, Vega20c_r, Wistia, Wistia_r, YlGn, YlGnBu, YlGnBu_r, YlGn_r, YlOrBr, YlOrBr_r, YlOrRd, YlOrRd_r, afmhot, afmhot_r, autumn, autumn_r, binary, binary_r, bone, bone_r, brg, brg_r, bwr, bwr_r, cool, cool_r, coolwarm, coolwarm_r, copper, copper_r, cubehelix, cubehelix_r, flag, flag_r, gist_earth, gist_earth_r, gist_gray, gist_gray_r, gist_heat, gist_heat_r, gist_ncar, gist_ncar_r, gist_rainbow, gist_rainbow_r, gist_stern, gist_stern_r, gist_yarg, gist_yarg_r, gnuplot, gnuplot2, gnuplot2_r, gnuplot_r, gray, gray_r, hot, hot_r, hsv, hsv_r, icefire, icefire_r, inferno, inferno_r, jet, jet_r, magma, magma_r, mako, mako_r, nipy_spectral, nipy_spectral_r, ocean, ocean_r, pink, pink_r, plasma, plasma_r, prism, prism_r, rainbow, rainbow_r, rocket, rocket_r, seismic, seismic_r, spectral, spectral_r, spring, spring_r, summer, summer_r, tab10, tab10_r, tab20, tab20_r, tab20b, tab20b_r, tab20c, tab20c_r, terrain, terrain_r, viridis, viridis_r, vlag, vlag_r, winter, winter_r
• Run Correlation heatmap and different color palettes CMAP. Possible values for the colormap cmap are-
  Accent, Accent_r, Blues, Blues_r, BrBG, BrBG_r, BuGn, BuGn_r, BuPu, BuPu_r, CMRmap, CMRmap_r, Dark2, Dark2_r, GnBu, GnBu_r, Greens, Greens_r, Greys, Greys_r, OrRd, OrRd_r, Oranges, Oranges_r, PRGn, PRGn_r, Paired, Paired_r, Pastel1, Pastel1_r, Pastel2, Pastel2_r, PiYG, PiYG_r, PuBu, PuBuGn, PuBuGn_r, PuBu_r, PuOr, PuOr_r, PuRd, PuRd_r, Purples, Purples_r, RdBu, RdBu_r, RdGy, RdGy_r, RdPu, RdPu_r, RdYlBu, RdYlBu_r, RdYlGn, RdYlGn_r, Reds, Reds_r, Set1, Set1_r, Set2, Set2_r, Set3, Set3_r, Spectral, Spectral_r, Vega10, Vega10_r, Vega20, Vega20_r, Vega20b, Vega20b_r, Vega20c, Vega20c_r, Wistia, Wistia_r, YlGn, YlGnBu, YlGnBu_r, YlGn_r, YlOrBr, YlOrBr_r, YlOrRd, YlOrRd_r, afmhot, afmhot_r, autumn, autumn_r, binary, binary_r, bone, bone_r, brg, brg_r, bwr, bwr_r, cool, cool_r, coolwarm, coolwarm_r, copper, copper_r, cubehelix, cubehelix_r, flag, flag_r, gist_earth, gist_earth_r, gist_gray, gist_gray_r, gist_heat, gist_heat_r, gist_ncar, gist_ncar_r, gist_rainbow, gist_rainbow_r, gist_stern, gist_stern_r, gist_yarg, gist_yarg_r, gnuplot, gnuplot2, gnuplot2_r, gnuplot_r, gray, gray_r, hot, hot_r, hsv, hsv_r, icefire, icefire_r, inferno, inferno_r, jet, jet_r, magma, magma_r, mako, mako_r, nipy_spectral, nipy_spectral_r, ocean, ocean_r, pink, pink_r, plasma, plasma_r, prism, prism_r, rainbow, rainbow_r, rocket, rocket_r, seismic, seismic_r, spectral, spectral_r, spring, spring_r, summer, summer_r, tab10, tab10_r, tab20, tab20_r, tab20b, tab20b_r, tab20c, tab20c_r, terrain, terrain_r, viridis, viridis_r, vlag, vlag_r, winter, winter_r
• Do dummy coding using ‘for loop’
• Create features and labels for decision tree regression using ‘.drop’
• Import Decision Tree Regression object from sklearn and set the minimum leaf size to 30. Fit the tree on overall data
• Visualize the Tree using graphviz within the jupyter notebook and also import the decision tress as pdf using ‘.render’
• Find out the predicted values using the tree

As you can see from the above decision tree, Limit, Income and Rating come out as the most important variables in predicting the “Balances/Card”.

The highest balance is for the customers who have credit limit more than $6,232 and income less than $ 69K. This makes sense as these people have higher lines available for them to buy items on and at the same time have lower income that prompts them to borrow more.

Thanks for reading! Please don’t forget to like and share with others!!

What is Support Vector Machine?

Support Vector Machine are supervised machine learning algorithms used mainly for classification and regression tasks. If a SVM is used for classification, it’s called Support Vector Classifier (SVC). Similarly, for regression it’s called Support Vector Regressor (SVR)

Where is SVM used?

SVM can be used wherever we use other machine learning techniques such as Logistic Regression, Decision Trees, Linear Regression, Naive Bayes Classifier etc. However, SVM may be particularly more suitable for following cases-

• Sparse data
• High Dimensional data
• Text Classification
• Data is nonlinear
• Image classification
• Data has complex patterns
• Etc.

How does an SVM work?

A support vector machine, works to separate the pattern in the data by drawing a linear separable hyperplane in high dimensional space. For example in the 2D image below, we need to separate the green points from the red points. We can draw many hyperplanes such as H1, H2, H3 and H4. They all help in separating the points.

However, since there are many possible hyperplanes as denoted in the image below, which hyperplane should be chosen? The answer is- the plane which maximizes the separation between the green and the red points. In this case, it happens to be H3.

What happens if the data is not linearly separable?

Kernel trick or Kernel function helps transform the original non-linearly separable data into a higher dimension space where it can be linearly transformed. See image below-

What is the best hyperplane?

As we discussed earlier, the best hyperplane is the one that maximizes the distance (you can think about the width of the road) between the classes as shown below

How to build this in Python?

Here is an excellent link for the hyper-parameter optimisation-

https://medium.com/all-things-ai/in-depth-parameter-tuning-for-svc-758215394769

For more info on sklearn library, refer below links-

http://scikit-learn.org/stable/auto_examples/svm/plot_rbf_parameters.html

http://scikit-learn.org/stable/modules/svm.html

Thanks for reading!

 

Here are the step by step directions on how to open Jupyter Notebook in the browser of your preference.

Step 1- Go to Anaconda Navigator and start Jupyter Notebook

Step 2- Go to Ananconda Prompt to grab URL

Step 3- Put the URL (will be unique for your application) in browser of your choice

That’s it. You will have Jupyter notebook open in the browser of your choice.

For clearing up anaconda prompt screen, simply type – ‘cls’ on the command prompt

Thanks for reading!

What is BFSI?

• BFSI is an acronym for Banking, Financial Services and Insurance. This covers a whole gamut of activities and business models.
• Wiki defines – “ BFSI comprises commercial banks, insurance companies, non-banking financial companies, cooperatives, pensions funds, mutual funds and other smaller financial entities. Banking may include core banking, retail, private, corporate, investment, cards and the like ”

Activity- Explore the below pages. List down different products and services that you see?

• https://www.chase.com/
• https://www.creditkarma.com/
• http://www.boc.cn/en/
• https://www.hsbc.co.in/1/2/homepage#
• https://www.wellsfargo.com/
• https://paytm.com/
• https://www.icicibank.com/
• https://www.hdfcbank.com/
• https://www.banamex.com/en/
• https://www.onlinesbi.com/

Thank you!

Autoregressive Integrated Moving Average (ARIMA) is one of the most popular technique for time series modeling. This is also called Box-Jenkins method, named after the statisticians who pioneered some of the latest developments on this technique.

We will focus on following broad areas-

• What is a time series? We have covered this in another article. Click here
• Explore a time series data. Please refer to the slides 2 to 7 of the below deck and Click here 
• What is an ARIMA modeling
• Discuss stationarity of a time series
• Fit an ARIMA model, evaluate model’s accuracy and forecast for future

What is an ARIMA modeling-

An ARIMA model has following main components. However, not all models need to have all of the below mentioned components.

• Autoregressive (AR)

Value of a time series at time period t (yt) is a function of values of time series at previous time periods ‘p’

yt = Linear function of yt-1, yt-2,….., yt-p + error

• Integrated (I)

To make a time series stationary (discussed below), sometimes we need to difference successive observation and model that. This process is known as integration and differencing order is represented as ‘d’ in an ARIMA model.

• Moving Average (MA)

Value of a time series at time period t (yt) is a function of errors at previous time periods ‘q’

yt = Linear function of Et-1, Et-2,….., Et-q + error

Based on the combinations of the above factors, we can have following and other models-

• AR- Only autoregressives terms
• MA- Only moving averages terms
• ARMA- Both autoregressive and moving average terms
• ARIMA- Autoregressive, moving average terms and integration terms. After the differencing step, the model becomes ARMA

A general ARIMA model is represented as ARIMA(p,d,q) where p, d and q represent AR, Integrated and moving averages respectively. Whereas each of p,d and q are integers higher than or equal to zero.

Stationarity of a time series- 

A time series is called stationary where it has a constant mean and variance across the time period, i.e. mean and variance don’t depend on time. It other words, it should not have any trend and dispersion in variance of the data over a period of time. This is also called white noise.

Please refer to slides 8 to 11 of the below deck for live examples of this discussion

From the plot of our air passengers time series, we can tell that the time series is not stationary. Moreover, a time series needs to be stationary or made stationary before being fed into ARIMA modeling.

Statistically, Augmented Dickey–Fuller test is used for testing the stationarity of a time series. Generally speaking the null hypothesis (H0) is that the series is “Non-Stationary” and the alternative hypothesis (Ha) is that series is “Stationary”.

If the p statistics generated from the test is less than <0.05 we can reject the null hypothesis. Otherwise, we need to accept the null hypothesis.

From the ADF test we can see that the p values is close to 0.78 and which is more than 0.05 and hence we need to accept the null hypothesis that is the series is “Non Stationary”

How do we make a time series stationary? Well, we can do it two ways-

• Manual- Transformation and Differecing etc. Let’s look at an example.
• Automated- The Integrated term (d)in the ARIMA will make it stationary. This we will do in the model fitting phase. Generally speaking we don’t require d>1 to make a time series stationary
• Auto.arima ( ) will take care of this automatically and fit the best model

Fit a model, evaluate model’s accuracy and forecast

We will use auto.arima ( ) to fit the best model and evaluate model fitment and performance using following main parameters.

Please refer to slides 12-18 of the below deck

A good time series model should have following characteristics-

• Residuals shouldn’t show any trends over time.
• Auto correlation Factors(ACF) and Partial Auto correlation Factor (PACF) shouldn’t have large values (beyond significance level)for any lags. ACF indicates correlation between the current value to all the previous values in a range. PACF is an extension of ACF, where it removes the correlation of the intermediate lags. You can read more on this here.
• Errors shouldn’t show any seasonality
• Errors should be normally distributed
• Error (MAE, MAPE, MSE etc.) should be low
• AIC, BIC should be relatively lower compared to other alternative models.

The codes and presentation 

For those who would like to read more about the time series analysis in R, here is an excellent free book.

Thank you!

What is a time series?

When we track a certain variable over an interval of time (generally at an equal interval of time) the resulting process is called a time series.

Let’s Look at some examples of time series in our daily life

1. Closing price of Apple stock on a daily basis will be a time series

2. GDP of the world over last several decades so will again be a time series again-

3. Similarly, the hourly movement of the Bitcoin prices in a day will be a time series

As you can see from the above examples, the duration of the time can vary for a time series. It can be minutes, days, hours, weeks. months, quarters, years or any other time period. However, one thing that will be common in all time series will be that a particular variable is being measured over a period of time.

What is a time series modeling?

A time series modelling is a statistical exercise where we try to achieve following two main objectives,

1. Visualize and understand the pattern of a particular time series. For example, if you are looking at the sales of an eCommerce company you would like to understand how it has performed over a period of time, which months it goes higher and lower etc.

2. By looking into the historical pattern, forecast what may happen in the future in that particular time series

What are the business usage of a time series modeling?

Time series modelling is used for a variety of different purpose. Some examples are listed below-

1. Forecast sales of an eCommerce company for the next quarter and next one year for financial planning and budgeting

2. Forecast call volume on a given day to efficiently plan resources in a call center

3. Predict trends in the future stock price movement for technical trading of that stock in a stock market

How is a time series forecasting different from a regression modeling?

One of the biggest difference between a time series and regression modeling is that a time series leverages the past value of the same variable to predict what is going to happen in the future.

On the other hand, a regression modeling such as a multiple linear regression will predict the value of a certain variable as a function of other variables

Let’s take an example to make this point more clear. If you are trying to protect the sales of an E-Commerce company as a function of what has been the sales in the past quarter this is a time series modelling

On the other hand, if you are trying to predict the sales of the same E-Commerce company as a function of other variables such as the marketing spend, price of the product and other such contributing factors, it is a regression modelling

What are the constituents of a time series?

A time series could be made up of following main parts

1. Trend- A systematic pattern of how the time series is behaving over a period of time. For example- GDP of emerging economies such as India is growing over a period of time

2. Seasonality- Peaks and troughs which happen during the same time. For example- sales of retailers in US goes higher during Thanksgiving and Black Friday

3. Random noise- As the name suggests, this is the random pattern in a time series

4. Cyclical- Cycles such as Fuel prices go low during certain time and higher at other times. Generally speaking a cycle is long in duration.

Please note that not all time series will have all these components.

Let’s look at example of the time series components. This has been done in R using the decompose function.

Additive Seasonal Model-

This model is used when the time series shows additive seasonality. For example, an eCommerce company sales in October of each year is $2MM USD higher than the base level sales regardless of what is the base level sales in that particular year. In very simplified mathematical equation it can be represented as

Observed = Trend + Seasonal + Random

Please take a look at the slide 2 and 3 of the below presentation

Multiplicative Seasonal Model-

This model is used when the time series shows multiplicative seasonality. For example, for an eCommerce company sales in October of each year is 1.2 times the base level sales in the year. If a particular year has low base level sales, the sales in October will be lower in absolute sense, however it will be 1.2x of the base level sales. In very simplified mathematical equation it can be represented as

Observed = Trend x Seasonal x Random

Please take a look a the slide 4 of the below presentation

Let’s now fit Exponential Smoothing to the above data example. Holt Winters is one of the most popular technique for doing exponential smoothing of a time series data. Moreover, we can fit both additive and multiplicative seasonal time series using HoltWinters() function in R.

There are many parameters that one can pass on this method, however one doesn’t need to pick these parameters as R will automatically pick the best settings to minimize the Square Error between the predicted and the actual values for the forecast.

The three most important parameters that one needs to pay attention to are-

alpha = Value of smoothing parameter for the base level.
beta = Value of smoothing parameter for the trend.
gamma = Value of smoothing parameter for the seasonal component.

All three of the above parameters range between 0 and 1

• If beta and gamma are both zero and alpha is non zero, this is known as Single Exponential Smoothing
• If gamma is zero but both beta and alpha are non zero, this is known as Double Exponential Smoothing with trend
• If all three of them are non zero, this is knows as Triple Exponential Smoothing or Holt Winters with trend and seasonality.

In the below example, we will let R choose the optimized parameters for us.

Additive Seasonal Holt Winters Model

Let’s fit an additive model first and compute MAE. The general form of an additive model is shown below.

yt = base + linear *  t + St + Random Error

Where

yt = forecast at time period t

base = Base signal

linear = linear trend component

t= time period t

St = Additive seasonal factor

This is the model that R has fitted for us-

HoltWinters(x = fl, seasonal = “additive”)

Smoothing parameters:
alpha: 0.2479595
beta : 0.03453373
gamma: 1

Coefficients:
[,1]
a 477.827781
b 3.127627
s1 -27.457685
s2 -54.692464
s3 -20.174608
s4 12.919120
s5 18.873607
s6 75.294426
s7 152.888368
s8 134.613464
s9 33.778349
s10 -18.379060
s11 -87.772408
s12 -45.827781

See Slide # 11 on how to use the above model output to compute forecast for any given time period. However, you don’t have to do it by hand as R will do it for you. Nevertheless, good to know how to use the above model output.

Finally, let’s notice that MAE of the additive model comes out to be 9.774438

Multiplicative Seasonal Holt Winters Model

The general form of a multiplicative model is shown below-

yt = (base + linear *  t )* St + Random Error

Where

yt = forecast at time period t

base = Base signal

linear = linear trend component

t= time period t

St = Additive seasonal factor

This is the model that R has fitted for us-

Call:
HoltWinters(x = fl, seasonal = “multiplicative”)

Smoothing parameters:
alpha: 0.2755925
beta : 0.03269295
gamma: 0.8707292

Coefficients:
[,1]
a 469.3232206
b 3.0215391
s1 0.9464611
s2 0.8829239
s3 0.9717369
s4 1.0304825
s5 1.0476884
s6 1.1805272
s7 1.3590778
s8 1.3331706
s9 1.1083381
s10 0.9868813
s11 0.8361333
s12 0.9209877

As you can see from the above output, the seasonality shows that demand for the air travel is the highest in July and August of each year and lowest in November.

Moreover the MAE for this model is 8.393662. Therefore, in this case a multiplicative Holt Winters seasonal model is able to provide us a better forecast compared to an additive model.

All the codes and output can be found here and in the below presentation.

Here is the forecast generated from the model-

You can do the HoltWinters Forecast in Excel as well using the below simple steps-

Thank you!

After you have built a machine learning model which is doing a great job in prediction, you don’t have to retrain your model again and again for future usage. Instead, you can use Python pickle serialization for reusing this model in future and transferring it into a production environment where non modelers can also use this model to make predictions.

 

First let’s look at how Wikipedia defines a pickle

Pickling is the process of preserving or expanding the lifespan of food by either anaerobic fermentation in brine or immersion in vinegar. The resulting food is called a pickle.

Python pickling is the same process without brine or vinegar, whereas you will pickle your model for longer usage without the need for you to recook your models. In a “Pickling” process a Python object is converted into a byte stream. On the other hand, in an “Unpickling” process a byte stream is converted back into Python object.

I strongly recommend that you read Python Official Documentation on this topic before moving forward.

Now let’s see this live in action. We will first look at a simple example and then look at a model example.

Example 1- In this we will pickle and un-pickle a simple Python list

Example 2- In this we will pickle and un-pickle a Decision Tree classifier and use it later for making predictions on a new data

For more details, do check out this excellent presentation.

Cheers!