Reference no: EM132320255 , Length: 15

Multivariate Analysis for High-Dimensional Data Assignment - Final Project

Instructions - There are two parts to this assessment. Part A is the project analysis of data as described in detail on pages 2-4 of this document. Your submission will be no more than 6-9 pages for Part A. Part B is the completion of a set of 4 questions on page 5-6 of this document. Additionally, you will submit one R script file for your work in Part A.

Part A -

Objective: The purpose of this project is to provide you with an opportunity to demonstrate an advanced level of synthesis, understanding and communication of the concepts, statistical methods and practical analyses within R that you have learnt throughout this course.

The Data: This data comes from a water quality study [1] where samples were taken from sites on different European rivers during a period of approximately one year. These samples were analysed for various chemical substances and, in parallel, algae samples were collected to determine the algae population distributions for seven algal species.

The impact on the environment of toxic waste, from a wide variety of manufacturing processes, is well known. It has also become clear that the subtler effects of nutrient level and chemical balance changes arising from farming land run-off and sewage water treatment also have a serious, but indirect, effect on the states of rivers, lakes and even the sea. Research has focused on the relationships among groups of chemical variables, among algae species and between both chemical and biological variables. The influence of season, river size and river flow rates are also considered important.

There are a total of 200 cases (rows/rivers) each containing 18 values (columns/variables). The first 3 variables in the data set are the season, river size, and fluid velocity of the river. The next eight variables are the chemical concentrations which should be relevant for the algae distribution: pH (measured on of scale of 1 to 14) and nitrogen, nitrates, nitrites, ammonia, phosphate, oxygen and chloride (all measured in mg/L). The last seven values of each row are the amount of different kinds of algae.

These 7 kinds are only a very small part of the whole algae community. The value 0.0 means that the frequency is very low. The data set also contains some missing data which are labelled with the string XXXXX.

A consultancy firm has asked you to explore this data and address three specific aspects of interest (Tasks 1, 2 and 3 below) for their client, and then report your process (what you have done and why) and findings in a written report. Before beginning the Tasks, you may need to do some data cleaning due to missing data or outliers. All analysis for the following tasks should be based on your cleaned data set and the structure/characteristics of your final data set should be well defined.

For this exercise assume that the data meets any required MVN assumptions (do not test for UVN or MVN).

The Tasks:

Task 1 - The client would like to know the number of rivers in the sample after cleaning. In addition, the number of rivers measured in each season, and each river size is required and some appropriate summary statistics/plots for each of the 8 chemical variables individually.

Action: Clean the data as you think necessary and then provide a frequency table of the number of rivers measured in each season, and then each river size. Determine appropriate summary statistics for each of the 8 chemical variables and the best way to present this information. Interpret interesting aspects of this data summary.

They would also like to know what the relationships are between the combination of river size & velocity based on the chemical variables. Which groups are most similar, and which are most different?

Action: First, create a new variable called 'river_size_vel' with categories that are combinations of the 3 River_Size and 3 River_Vel categories and provide a frequency table of the number of rivers in each new category. To show the multivariate relationships among the categories of the new 'river_size_vel' variable present the dendrogram and an MDS plot that best represent the relationships - as part of your interpretation explain what types of distances and clustering methods you have used and why.

Task 2 - The client would like to know if there are significant differences among the four seasons in terms of average river health as indicated by all of the chemical and algae variables?

Action: Select the best method (from those covered in this course only) to explore this question, perform the analysis and interpret. Include in your answer appropriate p-values for all significance tests performed.

Task 3 - The client would like to know if the season can be predicted based on all chemical and algae variables. The client is only interested in data related to Autumn, Spring and Summer?

Action: Select the best method (from those covered in this course only) to explore this question, perform the analysis, explain all relevant details of your process and interpret the results.

Instructions -

You will submit ONE pdf file for your project which will include Part A of your report addressing Task 1, 2 and 3 and Part B. All sections of Part A should be clearly labelled. You will also submit ONE R script file with all analysis code for Part A clearly labelled and commented.

You should not include any R code in your pdf report submission.

For each of Task 1, 2 and 3 your report should be no more than 2-3 pages (i.e. no more than 9 pages total). Any additional pages for Part A will not be marked. This means you will need to be very concise and clear in the information you provide in your final report.

Things to include for each of Task 1, 2 and 3:

• Was any data cleaning necessary (removal of cases with missing data, outliers, etc.)?
• Did you need to subset, rearrange or summarise the data in order to complete your analysis?
• What analysis was performed and what specific choices were made in the analysis process that would be necessary to know for the analysis to be repeated?
• What are the important aspects of the results to convey and did the analysis successfully address the aims of the company management?
• Are there any caveats/limitations you would place on the results or suggestions for future analysis?
• Do the results for each Task relate to previous Tasks?

Part B -

Include your responses to these Part B questions at the end of your Part A pdf submission (i.e. only one pdf file to be submitted for this whole project assessment item)

Question 1 - Recreate and complete the table below by indicating which features are relevant to each method.

Question 2 - Construct, by hand, a simple nearest-neighbour dendrogram from the distance matrix below. Do not produce the dendrogram in R. Use the distances to 'sketch' the relationships

Question 3 - Calculated by hand the Euclidian distance between individuals 1 and 2 for variables X1 and X2. Show all working.

Question 4 - What are some limitations or disadvantages of multivariate methods generally? (no more than 300 words).

Question 5 - Explain your understanding of eigen vectors and eigen values (your answer must be in your own words and will be checked using a plagiarism checker) (no more than 300 words).

Question 6 - Based on the Parallel Analysis table below, how many factors would you interpret? Explain you answer.

Factor Actual eigen value 95 percentile

Attachment:- Assignment File.rar