Write the lexical analyzer function and some test code

Assignment Help Computer Engineering
Reference no: EM132249212

Assignment -

For Program, the lexical analyzer, you will be provided with a description of the lexical syntax of the language. You will produce a lexical analysis function and a program to test it.

The lexical analyzer function must have the following calling signature:

Token getNextToken(istream& in, int& linenumber);

The first argument to getNextToken is a reference to an istream that the function should read from. The second argument to getNextToken is a reference to an integer that contains the current line number. getNextToken will update this integer every time it reads a newline. getNextToken returns a Token. A Token is a class that contains a TokenType, a string for the lexeme, and the line number that the token was found on.

A header file, tokens.h, will be provided for you. It contains a declaration for the Token class, and a declaration for all of the TokenType values. You MUST use the header file that is provided. You may NOT change it.

The lexical rules of the language are as follows:

1. The language has identifiers, which are defined to be a letter followed by zero or more letters or numbers. This will be the TokenType ID.

2. The language has integer constants, which are defined to be one or more digits. This will be the TokenType ICONST.

3. The language has string constants, which are a double-quoted sequence of characters, all on the same line. This will be the TokenType SCONST.

4. A string constant can include escape sequences: a backslash followed by a character. The sequence \n should be interpreted as a newline. The sequence \\ should be interpreted as a backslash. All other escapes should simply be interpreted as the character after the backslash.

5. The language has reserved the keywords print, set, if, loop, begin, end. They will be TokenTypes PRINT SET IF LOOP BEGIN END.

6. The language has several operators. They are + - * / ( ) which will be TokenTypes PLUS MINUS STAR SLASH LPAREN RPAREN.

7. The language recognizes a semicolon as the token SC.

8. The language recognizes a newline as the token NL.

9. A comment is all characters from a # to the end of the line; it is ignored and is not returned as a token. NOTE that a # in the middle of an SCONST is NOT a comment!

10. Whitespace between tokens can be used for readability. It serves to delimit tokens.

11. An error will be denoted by the ERR token.

12. End of file will be denoted by the DONE token.

Note that any error detected by the lexical analyzer should result in the ERR token, with the lexeme value equal to the string recognized when the error was detected.

Note also that both ERR and DONE are unrecoverable. Once the getNextToken function returns a Token for either of these token types, you shouldn't call getNextToken again.

The assignment is to write the lexical analyzer function and some test code around it.

It is a good idea to implement the lexical analyzer in one source file, and the main test program in another source file.

The test code is a main() program that takes several command line arguments:

  • v (optional) if present, every token is printed when it is seen.
  • strings (optional) if present, print out all the string constants in alphabetical order.
  • ids (optional) if present, print out all of the identifiers in alphabetical order filename (optional) if present, read from the filename; otherwise read from standard in.

Note that no other flags (arguments that begin with a dash) are permitted. If an unrecognized flag is present, the program should print "UNRECOGNIZED FLAG {arg}", where {arg} is whatever flag was given, and it should stop running.

At most one filename can be provided, and it must be the last command line argument. If more than one filename is provided, the program should print "ONLY ONE FILE NAME ALLOWED" and it should stop running.

If the program cannot open a filename that is given, the program should print "CANNOT OPEN {arg}", where {arg} is the filename given, and it should stop running.

The program should repeatedly call getNextToken until it returns DONE or ERR. If it returns DONE, the program proceeds to handling the -strings and -ids options, in that order. It should then print summary information and exit.

If getNextToken returns ERR, the program should print "Error on line N ({lexeme})", where N is the line number for the token and lexeme is the lexeme from the token, and it should stop running.

If the -v option is present, the program should print each token as it is read and recognized, one token per line. The output format for the token is the token name in all capital letters (for example, the token LPAREN should be printed out as the string LPAREN. In the case of token ID, ICONST, and SCONST, the token name should be followed by a space and the lexeme in parens. For example, if the identifier "hello" is recognized, the -v output for it would be ID (hello).

The -strings option should cause the program to print STRINGS: on a line by itself, followed by every string constant found, one string per line, in alphabetical order. If there are no SCONSTs in the input, then nothing is printed.

The -ids option should cause the program to print IDENTIFIERS: followed by a comma-separated list of every identifier found, in alphabetical order. If there are no IDs in the input, then nothing is printed.

The summary information is as follows:

Total lines: L

Total tokens: N

Where L is the number of input lines and N is the number of tokens (not counting DONE).

If L is zero, no further lines are printed.

PART 1 -

  • Compiles
  • Argument error cases
  • Files that cannot be opened
  • Too many filenames
  • Properly handles a zero length file
  • Recognizes keywords and identifiers
  • Summary information
  • -v mode

PART 2 -

  • Recognizes all remaining tokens.
  • Recognizes string with a newline in it as an error.
  • Recognizes string with a # in it as a string, not a comment.
  • Recognizes single character token types.
  • Supports -strings and -ids.

Reference no: EM132249212

Questions Cloud

Influencing generalized anxiety disorder : How might the environment interact with a person's biological systems in influencing generalized anxiety disorder?
Why is rifling present in the bore of the barrel : Why is rifling present in the bore of the barrel? What is the purpose of rifling? Compare and contrast a revolver versus a semiautomatic handgun.
Impact on generalized anxiety disorder : Do environmental influences have more of an impact on generalized anxiety disorder than biological influences or is genetics and biology most important?
How will end users typically respond to such announcements : How might an organization notify its users that all communications are being monitored and preserved?
Write the lexical analyzer function and some test code : The assignment is to write the lexical analyzer function and some test code around it. Note that no other flags (arguments that begin with a dash) are permitted
Identify and explain the organizations current global issue : ITS832: XYZ Company is a manufacturing company that produces various products and uses a supply chain to deliver directly to global customers and retailers.
Positive reinforcement function : A self-management strategy related to the positive reinforcement function has the goal of developing behavior that is under the control of verbal statements
How technology tools and policies have changed : Address how technology tools and policies have changed the legal landscape in the United States in a 3-4-page paper: use the following topics for the paper.
Anova logic-sum of squares : Data: The within-group (error) sum of squares is 108.45 and the between-group sum of squares is 48.68.

Reviews

Write a Review

Computer Engineering Questions & Answers

  Mathematics in computing

Binary search tree, and postorder and preorder traversal Determine the shortest path in Graph

  Ict governance

ICT is defined as the term of Information and communication technologies, it is diverse set of technical tools and resources used by the government agencies to communicate and produce, circulate, store, and manage all information.

  Implementation of memory management

Assignment covers the following eight topics and explore the implementation of memory management, processes and threads.

  Realize business and organizational data storage

Realize business and organizational data storage and fast access times are much more important than they have ever been. Compare and contrast magnetic tapes, magnetic disks, optical discs

  What is the protocol overhead

What are the advantages of using a compiled language over an interpreted one? Under what circumstances would you select to use an interpreted language?

  Implementation of memory management

Paper describes about memory management. How memory is used in executing programs and its critical support for applications.

  Define open and closed loop control systems

Define open and closed loop cotrol systems.Explain difference between time varying and time invariant control system wth suitable example.

  Prepare a proposal to deploy windows server

Prepare a proposal to deploy Windows Server onto an existing network based on the provided scenario.

  Security policy document project

Analyze security requirements and develop a security policy

  Write a procedure that produces independent stack objects

Write a procedure (make-stack) that produces independent stack objects, using a message-passing style, e.g.

  Define a suitable functional unit

Define a suitable functional unit for a comparative study between two different types of paint.

  Calculate yield to maturity and bond prices

Calculate yield to maturity (YTM) and bond prices

Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd