Reference no: EM133958718

Review Questions for Advanced Data Analytics

Data Preparation:
Load movie reviews data set aclIMDB.csv into a volatile corpus and then use "tm" and "SnowBallC", and "wordcloud" packages to clean the text: turn to small cases, remove numbers, remove punctuations, remove stop words, and stem to root words, and strip white spaces. Finally create DocumentTermMatrix and draw a word cloud for words with frequence over 20

Use EBImage package to load ten of your own photos into a 4-D array in the shape for CNN data inputs: batch x width x height x depth
Write a function that converts any time series into an array for RNN: batch x time steps x features.

Concepts checking:
Compute the entropy of the following set of labels: {S, F, S, S, F, F, F}
Compute the log-likelihood of the following observations given the binomial distribution parameter p = 0.05: {0, 1, 1, 1, 0}
Compute the likelihood of the following observations given a normal distribution with mean = 10 and standard deviation 2: 12.5, 11, 15, 23
Compute the binary cross-entropy and residual deviance of the following predictions of the logistic classifier and the actual labels:

Predicted Probability Actual Label
0.6 1
0.8 0
0.9 1
0.7 1
0.2 0

Compute precision, recall, sensitivity, accuracy, F1, and Keppa based on the following confusion matrix:

Predicted Positive Predicted Negative
Actual Positive 577 39
Actual Negative 15 890

Given the following table shows the number of messages that has word "Viagra" respectively for spam and ham messages. According to the data, what is a prior probability that a message is a spam? What is the likelihood of seeing word "Viagra" if a message is a ham? What is what is the posterior probability that a message is a spam if you see "Viagra" in the message?

Has word Viagra No word Viagra
Spam messages 100 900
Ham message 5 2000

Compute the output shape and the number of parameters for the following neural network layers:
m = keras_model_sequential() %>%
layer_simple_rnn(input_shape = c(5,10), units = 128, return_sequence = T)
m = kerals_model_sequential() %>%
layer_conv_2d(filters = 64, kernel_size = c(3,3), input_shape = c(28,28,3))
m = keras_model_sequential() %>%
layer_lstm(input_shape = c(5, 10), units = 64, return_state = T)
m = keras_model_sequential() %>%
layer_dense(units = 128, input_shape = 6)
Write the joint probability distribution as the product of marginal and conditional distributions for the following belief network

What is a backward and forward probability for a hidden Markov model? The following table shows forward probability of observing the test observations assuming the HMM model is for a promotor sequence. What is the likelihood that we can observe the test observations given the model?

The following is a Bayesian network. Perform the following tasks:

Create a moral graph and then determine, given that a person has lung cancer, whether smoking and dyspnoea are conditionally independent. Get AI-free online assignment help from experienced academic experts.
Create and visualize the network using graphviz package
Use data set clinic.csv to fit the network.
Use gRain package to compute the probability that a person has lung cancer given a positive x-ray?
Use the lizards dataset in bnlearn package to perform the following tasks:
Use hc() function to learn the structure of a Bayesian network and plot the network
Use the same dataset to perform a mantelhaen.test to see if Height and Diameter are conditional independent given Species.
*Suppose that you are to deal with the situation described in the following table, which gives the history of past production runs in a factory setting. Use C5.0 algorithm to select the best feature to be used to split outcome values in the first round toward building a decision tree. (Do the first level split only, and no need to finish the entire tree)

Supervisor Operator Machine Overtime Output
Patrick Joe A No high
Patrick Samantha B Yes low
Thomas Jim B Yes low
Patrick Jim B No high
Sally Joe C No high
Thomas Samantha C No low
Thomas Joe C No low
Patrick Jim A Yes low
The vectors to creating the data frame are included for your convenience:
supervisor = c("Patrick", "Patrick", "Thomas", "Patrick", "Sally", "Thomas", "Thomas", "Patrick")
operator = c("Joe", "Samantha", "Jim", "Jim", "Joe", "Samantha", "Joe", "Jim")
machine = c("A", "B", "B", "B", "C", "C", "C", "A")
overtime = c("No", "Yes", "Yes", "No", "No", "No", "No", "Yes")
output = c("high", "low", "low", "high", "high", "low", "low", "low")

Based on the "iris" dataset to create a logistic regression.
Create a new column, called output, in the dataset that takes value 1 if Species is setosa, and 0 otherwise.
Randomly select 80% of the cases in iris into training set and the rest into the testing set

Using the training dataset to train a logistic regression model to predict output against Sepal.Length as the predictor. Show the model summary and explain whether the result is good or bad by explaining two numerical values in the result
Draw the ROC curve for the above prediction and explain why or why not the regression is good or bad.

Use "insurance.csv" and create a neural network for linear regression problem to predict medical expenses with the following four layers: a dense layer of 128 neurons and "relu" activation, a dropout layer with 0.25 rate, a dense layer with 64 neurons with "relu" activation, a dropout layer with 0.5 rate, and an output layer with one neuron with linear activation.

Write code:

Write R code to normalize each column in a data frame into values between 0 and 1 if the column is numeric.

Write a function that can create a confusion matrix based a vector of labels and a vector of classifications and return precision, recall, F1, Kappa, accuracy using a list.

Write a function that can randomly pick a word from a list of words using a multinomial distribution, which is randomly generated by a Dirichlet distribution with a certain parameter alpha.

Write R code load all photos in a folder and put the image matrix into a 4-D array for a Convnet. (Hint: files = paste(1:100, ".jpg", sep= "") will generate a list of file names like 1.jpg, 2.jpg, ..., 100.jpg. Or use files = list.files(pattern= ".jpg") to get all jpg files in your working directory. )

Support we want to use past m observations to predict the next n observations. Write a function to remove all non-numeric columns out and then convert the remaining numeric columns into two 3-D arrays for LSTM training so that every m sequential examples into one batch for training input and every sequential n steps for training output.