Skip to content
Felix Notes

NodeJS

  • Introduction
    • The choice of languages is more varied, as long as it takes in an HTTP request and spits out HTML
    • Popular: PHP, C#, Ruby, Python and Java
    • Back-end handles application logic and data management and works with applications, databases, and servers
    • Back-end interacts with the front end often through APIs
    • Back-end developers build APIs that front-end developers can use to integrate the client with the server-side
  • REpresentational State Transfer (REST)

REST

  • Architectural style for providing communication standards between devices on the web
  • Refer to individual data objects as resources
  • Every resource/resource collection has a unique address associated with it - its URL
  • Stateless and separate the concerns of client and server
  • Separation of client and server
    • Development of client and server can be independent as long as each side knows what format of messages to send to the other
    • With a REST interface, different clients hit the same REST endpoints, perform the same actions, and receive the same responses
  • Statelessness
    • The server does not need to know about the state of the client and vice versa
  • Communication between server and client
    • Clients send requests to retrieve or modify resources, servers send response to those requests
  • Making requests
    • an HTTP verb, which defines what kind of operation to perform
      • GET --- retrieve a specific resource (by id) or a collection of resources
      • POST --- create a new resource
      • PUT --- update a specific resource (by id)
      • DELETE --- remove a specific resource by id
    • a header, which allows the client to pass along information about the request
      • The type of content that the client is able to receive from the server → the Accept field
      • Follows MIME types
    • a path to a resource
      • In RESTful APIs, paths should be semantic to the client
      • Hierarchical and descriptive path such as fashionboutique.com/customers/223/orders/12
      • Accessing a single resource would need an id appended to the path
      • Referring to list or collections of resources does not entail adding an id
    • an optional message body containing data
  • Sending responses
    • Content types
      • When server sends a data payload to the client, there must be a content-type in the header of the response
      • Match with one of the options in the Accept field of the request
    • Response codes - Alert the client to info about the success of the operation

Response codes

  • HTTP request types
    • HTTP standard specifies two properties for request types: safety and idempotency
    • HTTP GET and HEAD requests should be safe
      • Taking no actions other than retrieval
      • Cause no side effects on the server
      • Adhering to RESTful principles in our API guarantees safe GET and HEAD requests
    • All HTTP requests except POST should be idempotent
      • The side effects of N > 0 identical requests are the same as for a single request
      • Adhering to RESTful principles in our API guarantees idempotent GET, HEAD, PUT, and DELETE requests
  • Same origin policy and CORS
    • A URL’s origin is defined as protocol (scheme) + hostname + port
      • http://example.com:80/index.html
      • protocol: http
      • host: example.com
      • port: 80
    • If the referenced resource (from the fetched HTML document) is fetched using a URL that does not have the same origin as the source HTML, the browser will have to check the Access-Control-Allow-origin response header
    • If it contains * or the URL of the source HTML, the browser will process the response, otherwise it will refuse to process it and throw an error
    • Same-origin policy is a security mechanism implemented by browsers to prevent session hijacking and other security vulnerabilities
    • Legitimate cross-origin requests are enabled with CORS
      • Allows restricted resources (images, stylesheets, scripts, iframes, and videos) to be requested from another domain outside the domain from which the resource was served
    • But by default, JS code that runs in a browser can only communicate with a server in the same origin
  • Backend web architecture
    • Components
      • The server. This is the computer that receives requests
        • Personal computer used as server during development
      • The app. This is the application running on the server that listens for requests, retrieves information from the database, and sends a response
        • Contains logic about how to respond to various requests based on the HTTP verb and the URI
        • Route is a pair of an HTTP verb and a URI
        • Routing is matching requests to appropriate routes/endpoints, which triggers handler functions to process and send a response
        • Some of these handler functions will be middleware
        • Middleware might modify the request object, query the DB, or otherwise process the incoming request
          • Ends by passing control to the next middleware function, rather than sending a response
          • One middleware ends the request-response cycle with a response to the client
        • Each route can have one or many handler functions that are executed when a request to that route is matched
      • The database. Databases are used to organize and persist data.
        • An interface to save data in a persistent way to memory
        • Reduces load on memory of main server and allows data persistence when server down
        • Requests to server might require DB queries
    • Web API
      • An interface created by the back-end
        • A collection of endpoints and the resources these endpoints expose
      • Defined by the routes it defines, and responses client can expect
      • One Web API can provide data from different front-ends
  • Backend frameworks
    • Common features
      • Work directly with HTTP requests and responses
        • Every “view” function (a request handler) receives an HttpRequest object and return an HttpResponse object
        • HttpRequest contains request information
        • HttpResponse contains the formatted output
      • Routing requests to appropriate handlers
        • Map URL patterns to specific handler functions
      • Interacting with DBs
        • Object-Relational Mapper (ORM) is an abstraction database layer that abstracts DB read, write, query, and delete operations
        • ORM benefits
          • Can change underlying DB implementation without changing the code that uses it
          • Validation of data can be implemented within the framework
        • The model is the object used to define the structure of a record
          • Specifies field types, which may provide field-level validation
          • Field definition help specify their maximum size, default values, selection list options, help text for documentation, label text for forms etc.
      • Supporting sessions and user authorization
      • Formatting output (HTML, JSON, XML)/Rendering data
        • Provides templating systems for HTML, JSON, or XML output
        • Placeholders in the templates will be replaced with values passed in from the view function when the page is rendered
      • Improving security against web attacks
  • Node.js
    • An asynchronous event-driven JavaScript runtime designed to build scalable network applications
      • JS runtime means that you can run JS from any machine without going through a web browser
      • Adds some functionality to browser-based JS
        • Read and write local files
        • Create HTTP connections
        • Listens to network requests
      • In the browser, you do not have access to Node APIs like filesystem access functionality
      • In Node, you do not have access to the DOM, or Web Platform APIs, or objects provided by the browser
    • Node assumes it’s always running in a dev environment
    • By setting NODE_ENV=production environment variable, you can signal Node that you are running in prod
      • Execute the command export NODE_ENV=production in the shell
      • Or put it in shell config file (e.g .bash_profile with the Bash shell)
      • Can also apply the environment variable by
        • NODE_ENV=production node app.js
    • TypeScript → TS compiler → JS → Node.js
    • Tell the OS which interpreter to use for running JS scripts with a “shebang” line
      • #!/usr/bin/env node
      • The script should have executable permissions
        • chmod u+x app.js
    • The process core module of Node provides the env property that hosts all environment variables that were set at the moment the process was started
      • USER_ID=239482 USER_KEY=foobar node app.js
        • Passes the user USER_ID and USER_KEY
      • process.env.USER_ID; // “239482”
      • process.env.USER_KEY; // “foobar”
        • Accesses the environment variables inside the script
      • Configure bash scripts to export variables in prod
      • Create an .env file in the root directory of project and use dotenv package to load them during runtime
        • # .env file
        • USER_ID=“239482”
        • USER_KEY=“foobar”
        • NODE_ENV=“development”
        • # .js file
        • require(‘dotenv’).config();
        • process.env.USER_ID; // “239482”
        • process.env.USER_KEY; // “foobar”
        • process.env.NODE_ENV; // “development”
    • Make HTTP requests with the Axios library or the HTTPS core module
    • Interact with the file system using the fs core module
    • URL browser-compatible class
    • Handle events with the events core module
    • Recommended project structure ├── index.js → starts the application ├── app.js → contains actual application, connect to DB ├── build │ └── … ├── controllers │ └── notes.js → define event handlers of routes in Router object, to be used as middleware ├── models │ └── note.js → define schema ├── package-lock.json ├── package.json ├── utils │ ├── config.js → handle environment variables │ ├── logger.js → wrap native log functions to save to file or send to external logging services │ └── middleware.js → custom middleware
  • Separate test, prod, and dev environments
    • Examples
      • “start”: “cross-env NODE_ENV=production node index.js”
      • “dev”: “cross-env NODE_ENV=development nodemon index.js”
      • “test”: “cross-env NODE_ENV=test jest —verbose —runInBand”
    • Use cross-env as production dependency to ensure cross-platform compatibility
    • Assess environment with process.env.NODE_ENV
  • Test API with Jest and supertest
  • Using BaaS for backend
    • Firebase services
      • Firebase Authentication to easily allow your users to sign into your app.
      • Cloud Firestore to save structured data on the cloud and get instant notification when data changes.
      • Cloud Storage for Firebase to save files in the cloud.
      • Firebase Hosting to host and serve your assets.
      • Firebase Cloud Messaging to send push notifications and display browser popup notifications.
      • Firebase Performance Monitoring to collect user performance data for your app.
    • Integrate Firebase to JS project
  • Databases
    • Types
      • Relational
      • Non-relational
  • Express
    • Provides mechanisms to
      • Write handlers for requests with different HTTP verbs at different URL paths (routes)
      • Integrate with “view” rendering engines in order to generate responses by inserting data into templates
      • Set common web application settings like the port to use for connecting, and the location of templates that are used for rendering the response
      • Add additional request processing “middleware” at any point within the request handling pipeline
    • Express passes control from one middleware/route to the next by using next()
      • next() would simply move onto the next middleware/route handler
      • next(<param>) would continue to the error handler middleware
      • Can register middleware to all routes/specific routes/specific operations
    • Express error handlers
    • Order of middleware loading
      • json-parser middleware should be among the very first
      • unknownEndpoint should be next to the last middleware, just before the error handler
    • A router object is an isolated instance of middleware and routes
      • Think of it as a mini-application capable only of performing middleware and routing functions
      • Every Express app has a built-in app router
    • Use express-async-errors to eliminate try-catch blocks while using async/await
      • If an exception occurs in an async route, the execution is automatically passed to the error handling middleware
    • User admin
      • In relational DBs, save user’s ID as foreign key for respective object
      • Can do all sorts of stuff in document DBs because they don’t support join queries
      • Schema-less DBs like Mongo require radical design decisions about data organization
    • Token authentication

Token authentication

  • User starts by logging in using a login form implemented with React
  • This causes the React code to send the username and the password to the server address /api/login as a HTTP POST request
  • If the username and the password are correct, the server generates a token that somehow identifies the logged-in user
  • The backend finds the user in the DB with the supplied username and compare password with stored passwordHash
  • The token contains the username and user id in an encrypted form, which can be decoded in authorized endpoints to obtain the original username and userid using the same secret string used to sign the token → do not need to include username and user id in every request, only need to login, get token and include token with every request
  • The token is signed digitally, making it impossible to falsify (with cryptographic means)
  • The backend responds with a status code indicating the operation was successful and returns the token with the response
  • The browser saves the token, for example to the state of a React application
  • When the user creates a new note (or does some other operation requiring identification), the React code sends the token to the server with the request
  • The server uses the token to identify the user
  • Can use express-jwt for multiple interfaces requiring identification
  • Token authentication must always be used over HTTPS
  • Problems of token-based authentication
    • API has a blind trust to the token holder
    • To revoke the access rights of the token holder - Limit the validity period of a token - Safer the shorter the period is - requires more re-logins - Save info about each token to backend DB and check for each API request if the access right corresponding to the token is still valid - Called server-side session - Increased complexity and performance hit - Common to save the session corresponding to a token to a key-value DB (Redis) - Token can just be a random string - Cookie can be used instead of Authorization header
  • Django

Django

  • Should use same DB for prod and dev
  • MongoDB
    • Stores data records as documents, gathered together in collections
    • A database stores one or more collections of documents
    • Document databases like Mongo are schemaless
  • Mongoose
    • High-level API to use MongoDB directly in JS code, better than the official MongoDB Node.js driver library
    • An object document mapper (ODM)
    • The idea of Mongoose is that data stored in DB is given a schema at the application level that defines the shape of the documents stored in any given collection
    • Saving JS objects as Mongo documents is straightforward
    • Schema
      • In Mongoose, everything is derived from a schema
      • Each schema maps to a MongoDB collection and defines the shape of the documents in that collection
      • Each key in the schema defines a property which will be cast to its associated SchemaType
      • Permitted SchemaTypes
        • String
        • Number
        • Date
        • Buffer
        • Boolean
        • Mixed
        • ObjectId
        • Array
        • Decimal128
        • Map
        • UUID
      • Schema — (compilation) → Model
    • Model
      • Fancy constructors compiled from Schema definitions, used to construct documents
      • An instance of a model is called a document
      • Models are responsible for creating and reading documents from the underlying MongoDB database
    • Comparison with OOP
      • Mongoose schemas in MongoDB can be compared to interfaces in programming languages
      • Schemas define the structure and properties that a document should have
      • They specify fields, types, default values, and validation rules
      • Schemas act as blueprints or contracts for documents
      • Mongoose models are similar to classes based on interfaces
      • Models represent collections in MongoDB and provide an interface to interact with the database
      • Models allow operations like create, read, update, and delete on documents
      • Documents in Mongoose are instances of models, similar to objects being instances of classes
      • You can create new documents using the model, set properties, and save them to the database
      • Various operations like updating fields, running queries, and executing aggregation pipelines can be performed on documents
      • Mongoose’s similarity to programming language constructs makes it intuitive for developers familiar with object-oriented concepts
      • It provides a structured way to work with MongoDB and documents using JavaScript
    • Provides validation as middleware
    • Ensure uniqueness of a field with mongoose-unique-validator
    • Joins in relational DBs are transactional, the state of the DB does not change during the time that the query is made
    • Join queries in Mongoose are implemented with multiple queries, which does not guarantee the consistence of the DB state
      • Use the populate method and include needed fields