Solving Malware Classification Task using Python

data analysis and visualization; machine learning; cybersecurity-related data analytics
My interests:
Topic is important because:
application of machine learning techniques for malware detection allows to keep pace with malware evolution and combat security threats more effectively compared to other methods.

Terms
Malware
software that is specifically designed to disrupt, damage, or gain unauthorized access to a computer system
Benign Ware
ordinary software without any malicious activity

Main Steps
Dataset collection
Building a machine learning model
Data reduction
01
02
03

Dataset collection
01.
With data collection, “the sooner the better”, is always the best answer.
—Marissa Mayer

Problem
Create a dataset with features that will help the system distinguish between good and bad files:
find files representing malicious and benign activity
extract features from these files and tabulate them

Solution
Found:
3077 binary malicious files
1952 binary benign files
collected from “VX Heavens Virus Collection”
collected on local PC

Solution
Extracted:
100 features from binary portable executable files (.exe, .dll, .sys, etc.) using “pefile” python module

Dataset reduction
02.
Redundancy is expensive but indispensable.
—Jane Jacobs

Problem
Select features that yield the most accurate results:
apply data reduction algorithms
obtain dataset with reduced dimensionality

Solution
Applied:
Feature importance technique based on Gini importance metric
Principal component analysis (PCA)
for input features with low correlation
for input features with high correlation

Solution
Obtained:
10 features with the highest scores; the higher, the more important the feature

Solution
Obtained:
reduced the dimensionality of the data from 8 to 2
Principal component 1 - 78.77% of the variance
Principal component 2 - 13.03% of the variance

Building a machine learning model
03.
What we want is a machine that can learn from experience.
—Alan Turing

Problem
Determine which file is malicious and which is benign:
apply a machine learning algorithm
split the data into training and validation sets

Solution
The data was split into:
5 equal folds
Each fold was used for both training and validation.

Solution
Applied:
Decision Trees Classifier algorithm.
Built Decision Tree.
Classification rate (accuracy score): 0.9371

Libraries & frameworks used
Pandas
Numpy
Pefile
Scikit-learn
Matplotlib
Math

Resources
Presentation template
M. Zubair Shafiq et al. (2009) PE-Miner: Mining Structural Information to Detect Malicious Executables in Realtime. In: Engin Kirda, Somesh Jha, Davide Balzarotti, eds. Recent Advances in Intrusion Detection, 12th International Symposium, Saint-Malo: Springer, pp. 121-141.
California State University (2021) Malware, Trojan, and Spyware. [online], available from: https://www.csuchico.edu/isec/stories/malware-trojans-spyware.shtml#:~:text=Malware%3A%20Malware%20is%20short%20for,access%20to%20a%20computer%20system.
[accessed 13 June 2021]