SIT 384 Data Analytics for Cyber Security Assignment 2 Trimester 1 2018

Objectives

•   To apply skills and knowledge acquired throughout the trimester in classification algorithms

  and machine learning process.

•   To rationalize the use of machine learning algorithms to effectively and efficiently process of

  data in big size.

•   To demonstrate ability to use R to perform email classification tasks that are common for

  corporate security analyst.

•   To scientifically conduct and document machine learning experiments for analytics

  purposes.

  SIT 384 Data Analytics for Cyber Security Assignment 2 Trimester 1 2018

• Due Date: 5pm, Friday, May 18, 2018

  This assignment consists of a report worth 20 marks. Delays caused by student’s own computer downtime cannot be accepted as a valid reason for late submission without penalty. Students must plan their work to allow for both scheduled and unscheduled downtime.

  Submission instructions: You must submit an electronic copy of all your assignment files via Cloud- Deakin. You must include both your report, source codes, necessary data files and optionally presentation file. Assignments will not be accepted through any other manner of submission. Students should note that email and paper based submissions will ordinarily be rejected.

  Special requirements to prove the originality of your work: On-campus students (B and G) are required to demonstrate the execution of your classification programs in R to your tutor in Week 10; Cloud students are required to attach a 3-5 minutes Video presentation to demonstrate how your R codes are executed to derive the claimed results. The video should be uploaded to a cloud storage (You can find out how to upload a video from https://video.deakin.edu.au/.) Failure to do so will result a delayed assessment of your submission. SIT 384 Data Analytics for Cyber Security Assignment 2 Trimester 1 2018

  Late submissions: Submissions received after the due date are penalized at a rate of 5% (out of the full mark) per day, no exceptions. Late submission after 5 days would be penalized at a rate of 100% out of the full mark. Close of submissions on the due date and each day thereafter for penalties will occur at 05:00 pm Australian Eastern Time (UTC +10 hours). Students outside of Victoria should note that the normal time zone in Victoria is UTC+10 hours. No extension will be granted.SIT 384 Data Analytics for Cyber Security Assignment 2 Trimester 1 2018

  It is the student’s responsibility to ensure that they understand the submission instructions. If you have ANY difficulties ask the Lecturer/Tutor for assistance (prior to the submission date).

  SIT 384 Data Analytics for Cyber Security Assignment 2 Trimester 1 2018

• Copying, Plagiarism Notice
  This is an individual assignment. You are not permitted to work as a part of a group when writing this assignment. The University’s policy on plagiarism can be viewed online at http://www.deakin.edu.au/students/study-support/referencing/plagiarism

SIT 384 Data Analytics for Cyber Security Assignment 2 Trimester 1 2018

Overview
The popularity of social media networks, such as Twitter, leads to an increasing number of spamming activities. Researchers employed various machine learning methods to detect Twitter spams. In this assignment, you are required to classify spam tweets by using provided datasets. The features have been extracted and clearly structured in JSON format. The extracted features can be categorized into two groups: user profile-based features and tweet content-based features as summarized in Table 1.

The provided training dataset and testing dataset are separately listed in Table 2 and Table 3. In testing dataset, we can find that the ratio of spam to non-spam is 1:1 in Dataset1, while the ratio is 1:19 in Dataset 2. In most of previous work, the testing datasets are nearly evenly distributed. However, in real world, there are only around 5% spam tweets in Twitter, which indicates that testing Dataset 2 simulates the real-world scenario. You are required to classify spam tweets, evaluate the classifiers’ performance and compare the Dataset 1 and Dataset 2 outcomes by conducting experiments.

SIT 384 Data Analytics for Cyber Security Assignment 2 Trimester 1 2018

Twitter Spam Detection Work Flow

Problem Statement
This is an individual assessment task. Each student is required to submit a report of approximately 2,000-2,500 words along with exhibits to support findings with respect to the provided spam and non-spam messages. This report should consist of:

•   Overview of classifiers and evaluation metrics
•   Construction of data sets, identification of features and the process of conducting

  classification

•   Technical findings of experiment results
•   Justified discussion of the performance evaluation outcomes for different classifiers

  To demonstrate your achievement of these goals, you must write a report of at least 2,000 words (2,500 words maximum). Your report should consist of the following chapters:

  1. A proper title which matches the contents of your report.
  2. Your name and Deakin student number in the author line.
  3. An executive summary which summarizes your findings. (You may find hints on writing good

     executive summaries from http://unilearning.uow.edu.au/report/4bi1.html.)

  4. An introduction chapter which lists the classification algorithms of your choice (at least 5

     algorithms), the features used for classification, the performance evaluation metrics (at least 5 evaluation metrics), the brief summary of your findings, and the organization of the rest of your report. (You may find hints on features used for classification from Twitter Developer Documentation https://dev.twitter.com/overview/api )

  5. A literature review chapter which surveys the latest academic papers regarding the classifiers and performance evaluation metrics of your choice. With respect to each classifier and performance evaluation metrics, you are advised to identify and cite at least one paper published by ACM and IEEE journals or conference proceedings. In addition, Your aim of this part of the report is to demonstrate deep and thorough understanding of the existing body of knowledge encompassing multiple classification techniques for security data analytics,

specifically, your argument should explain why machine learning algorithms should be used rather than human readers. (Please read through the hints on this web page before writing this chapter http://www.uq.edu.au/student-services/learning/literature-review.))

SIT 384 Data Analytics for Cyber Security Assignment 2 Trimester 1 2018

Need Help in yourSIT 384 Assignment?

6. Technical demonstration chapter which consists of fully explained screenshots when your experiments were conducted in R. That is, you should explain each step of the procedure of classification, and the performance results for your classifiers. Note, what classifiers you presented in literature review should be what you conduct experiments.
7. Performance evaluation chapter which evaluates the performance of classifiers. You should analyse each classifier’s performance with respect to the performance metrics of your choice. In addition, you should compare the performance results in terms of evaluation metrics, e.g., accuracy, false positive, recall, F-measure, speed and so on, for the selected classifiers and datasets.
8. A conclusions chapter which summarizes major findings of the study (You should use at least 5 evaluation metrics to evaluate the performance of classifiers and compare the performance of different classifiers. You can demonstrate your experiment results in the form of table and plots), discusses whether the results match your hypotheses prior to the experiments and recommends the best performing classification algorithm.
9. A bibliography list of all cited papers and other resources. You must use in-text citations in Harvard style and each citation must correspond to a bibliography entry. There must be no bibliography entries that are not cited in the report. (You should know the contents from this page http://www.deakin.edu.au/students/study-support/referencing/harvard.)
10. SIT 384 Data Analytics for Cyber Security Assignment 2 Trimester 1 2018