Develop an online document library

Assignment Help Computer Engineering
Reference no: EM132191487

Systems Engineering - Assignment Organization

Task description

You are required to develop an online **document library**.

Users are presented with an input form, where they can submit *documents* (e.g., books, poems, recipes) along with *metadata* (e.g., author, mime type, ISBN).

For the sake of simplicity, they can view *all* stored documents on a single page.

Hence, create an application with the following architecture.

Don't worry, in this repository you can find some Makefiles, Dockerfiles, configuration files and source code to get you started.

Nginx

Nginx is a web server that delivers static content in our architecture.

Static content comprises the landing page (index.html), JavaScript, css and font files located in ``nginx/www``.

1. complete the ``nginx/Dockerfile``

a) upgrade the system
#) install nginx
#) copy ``nginx/nginx.conf`` from host to container's ``/etc/nginx/nginx.conf``
#) use port 80 in the container
#) run nginx on container startup

#. in docker-compose

a) build the image
#) assign nginx to the ``se_backend`` network
#) mount the host directory ``nginx/www`` to ``/var/www/nginx`` in the container

#. verify your setup (it should display the landing page)

HBase
~~~~~

We use HBase, the open source implementation of Bigtable, as database.
``hbase/hbase_init.txt`` creates the ``se2`` namespace and a ``library`` table with two column families: ``document`` and ``metadata``.

1. build the image for the container description located in ``hbase/``
#. in docker-compose

a) add hbase to the ``se_backend`` network

The Dockerfile exposes different ports for different APIs.
We recommend the JSON REST API, but choose whatever API suits you best.

1. `HBase REST documentation <http://hbase.apache.org/book.html#_rest>`_
#. the client port for REST is 8080
#. employ curl to explore the API

a) ``curl -vi -X PUT -H "Content-Type: application/json" -d '<json row description>' "localhost:8080/se2:library/fakerow"``
#) yes, it's really *fakerow*

#. ``gserve/src/gserve/HbaseJSON.go`` contains helpers to convert data from frontend JSON via Go types to base64-encoded HBase JSON and back
#. you might want to use the (Un)marshal functions from the `encoding/JSON package <https://golang.org/pkg/encoding/json/>`_

ZooKeeper
~~~~~~~~~

Deviating from the architecture image, you don't need to create an extra ZooKeeper container.
The HBase image above already contains a ZooKeeper installation.

1. add an alias to the hbase section in docker-compose such that other containers can connect to it by referring to the name ``zookeeper``

.. note::

1. you are allowed to use the `go-zookeeper <https://github.com/samuel/go-zookeeper>`_ library

grproxy
~~~~~~~

This is the first service/server you have to write by yourself.
Implement a reverse proxy that forwards every request to nginx, except those with a "library" prefix in the path (e.g., ``http://host/library``).
Discover running gserve instances with the help of ZooKeeper and forward ``library`` requests in circular order among those instances (Round Robin).

1. complete ``grproxy/Dockerfile``
#. in docker-compose

a) build grproxy
#) add grproxy to both networks: ``se_frontend`` and ``se_backend``

.. note::

1. you are allowed to use `httputil.ReverseProxy <https://golang.org/pkg/net/http/httputil/>`_
2. you don't need to handle the case where an instance registered to ZooKeeper doesn't reply

gserve
~~~~~~

Gserve is the second service you need to implement, and it serves basically two purposes.
Firstly, it receives ``POST`` requests from the client (via grproxy) and adds or alters rows in HBase.
And secondly, it replies to ``GET`` requests with an HTML page displaying the contents of the whole document library.
It only receives requests from grproxy after it subscribed to ZooKeeper, and automatically unsubscribes from ZooKeeper if it shuts down or crashes.

1. gserve shall return all versions of HBase cells (see output sample above)
#. the returned HTML page must contain the string *"proudly served by gserve1"* (or gserve2, ...) without HTML tags in between
#. complete ``gserve/Dockerfile``
#. in docker-compose

a) build gserve
#) start two instances *gserve1* and *gserve2*
#) add both instances to ``se_backend``
#) make sure, that both instances start after hbase and grproxy
#) provide the names of the instances (gserve1, gserve2) via environmental variables

Hints
-----

* Start small, don't try to solve every problem at once.
* Test your components against single Docker containers (e.g., gserve with HBase container), and integrate them into docker-compose later on.
* The developer tools of your browser may help you to capture and analyse requests and responses.

Frequently Asked Questions
--------------------------

1. How do I use the JSON/Base64-encoding/(Un)Marshaling code?

.. code:: go

package main

import "encoding/json"

func main() {
// unencoded JSON bytes from landing page
// note: quotation marks need to be escaped with backslashes within Go strings: " -> \"
unencodedJSON := []byte("{\"Row\":[{\"key\":\"My first document\",\"Cell\":[{\"column\":\"document:Chapter 1\",\"$\":\"value:Once upon a time...\"},{\"column\":\"metadata:Author\",\"$\":\"value:The incredible me!\"}]}]}")
// convert JSON to Go objects
var unencodedRows RowsType
json.Unmarshal(unencodedJSON, &unencodedRows)
// encode fields in Go objects
encodedRows := unencodedRows.encode()
// convert encoded Go objects to JSON
encodedJSON, _ := json.Marshal(encodedRows)

println("unencoded:", string(unencodedJSON))
println("encoded:", string(encodedJSON))
}

/*
output:

unencoded: {"Row":[{"key":"My first document","Cell":[{"column":"document:Chapter 1","tiny_mce_markerquot;:"value:Once upon a time..."},{"column":"metadata:Author","tiny_mce_markerquot;:"value:The incredible me!"}]}]}
encoded: {"Row":[{"key":"TXkgZmlyc3QgZG9jdW1lbnQ=","Cell":[{"column":"ZG9jdW1lbnQ6Q2hhcHRlciAx","tiny_mce_markerquot;:"dmFsdWU6T25jZSB1cG9uIGEgdGltZS4uLg=="},{"column":"bWV0YWRhdGE6QXV0aG9y","tiny_mce_markerquot;:"dmFsdWU6VGhlIGluY3JlZGlibGUgbWUh"}]}]}
*/

#. Do I need a library to connect with HBase?

No, we recommend the REST interface. You might also consider using Thrift, but we haven't tested it.

#. Could you provide an example for an HBase scanner?

Yes, for the command line:

.. code:: bash

#!/usr/bin/bash

echo "get scanner"

scanner=`curl -si -X PUT \
-H "Accept: text/plain" \
-H "Content-Type: text/xml" \
-d '<Scanner batch="10"/>' \
"http://127.0.0.1:8080/se2:library/scanner/" | grep Location | sed "s/Location: //" | sed "s/\r//"`

echo $scanner

curl -si -H "Accept: application/json" "${scanner}"

echo "delete scanner"

curl -si -X DELETE -H "Accept: text/plain" "${scanner}"

#. What is meant by "build gserve"?

Build the docker image with docker compose, **not** the gserve binary.

Optional

You had a lot of fun and want more?
No problem!
Select a topic you're interested in, and enhance any of the components.
For instance, query single documents or rows, replace nginx with a web server written by yourself, improve the error handling in Grproxy, write test cases or in the worst case just beautify the HTML/CSS.
But keep in mind: your application *shall still conform to the task description*.

Attachment:- document library.rar

Reference no: EM132191487

Vulnerabilities in purchasing through debit card

Recognize any potential vulnerabilities in act of making a purchase through the debit card, and which area of the CIA triad they apply to.

Discuss the strengths of a gui and a scripting approach

Discuss the strengths and weakness of a GUI and a scripting approach. Should a combined approach be considered that uses both GUIs and scripts? Give your recommendations for t

Create class complex for working with complex numbers

modify class Complex for working with complex numbers of the form a + bi, where i is square root of -1. Your class must have two overloaded operators for adding and subtract

Explaining the role of cio

Assess the role of CIO within any health care organization and explain the key characteristics a CIO would require to possess to be effective in that position.

What medical record number would you expect to find

In a terminal digit filing system, you see the file guide. What medical record number would you expect to find to the right of that file guide? Which one of the following sta

Create a method that will perform the a division operation

Write down a method that will perform the a division operation (divide by) on the numbers passed to it in two variables (numerator, denominator) and outputs the results. Use

Creating database for company

If you were preparing to create a brand new database for a company, do you think there would be any value in the interviewing the users to determine out what colors and desi

Write a program that allows numbers to be entered

Write a program that allows numbers to be entered in decimal (base 10), hexadecimal (base 16), or binary (base 2), and then prints out the number again in all three bases.

Reviews

Write a Review

 
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd