<img src="/assets/event-loop-async-io-nodejs/cover.png" alt="cover image">
<h2>Introduction;</h2>
<p>Nodejs is a Javascript Runtine environment ,it execute Javascript on a Blocking single thread. but what makes the popularity of Nodejs is it's approche for handling Asyncronous operation in an Non-blocking way ,operations like writing to disc , networking ect.
The Piece of Software that allow nodejs to manage between synchronous and asynchronous operation is Called the Event Loop. Every Nodejs Developer needs to have a good understanding of these concepts in order to develop effeciant applications and avoid common  performance issues. in this blog post we try to introduce generale understandings about the Nodejs Event Loop and Async IO, if you want to deep dive into Nodejs please check the further resource section.</p>
<h2>What compose Nodejs environment</h2>
<img src="/assets/event-loop-async-io-nodejs/node-architecture.png" alt="cover image">
<p>what we see in the illustration above is an abstract way to define the composition of nodejs ,we have :</p>
<ul>
<li>
<p><strong>V8 Engine:</strong> is the part that compile javascript into machine code and then  execute it in a single thread  meaning that all the operation will be execute sequencialy one By one, it uses a <strong>call stack</strong> to keep track of functions calls and mmanaging their execution .</p>
</li>
<li>
<p><strong>Libuv</strong>: is a cross-platform library   responsible   for handling IO operations whether by calling OS handlers ,child process or using  threads Pool  whithout blocking the main thread . and abstracts the underlying operating system's file system, network, and concurrency capabilities. It handles tasks such as managing event loops, timers, file operations, networking, and threading. libuv allows Node.js to achieve non-blocking, event-driven behavior.</p>
</li>
<li>
<p><strong>native C/C++ modules</strong> : modules  like the 'http module' ,Dns resolver, bycrypt for  Encryption and Zlib for  compression etc.</p>
</li>
</ul>
<h2>The execution flow of execution :</h2>
<p>let's get back to the illustration mentioned before, we have a server application written in Node js and we launch it's execution ,what's really happend:</p>
<p>V8 Engine will compile the js code into machine code, prepare the Heap and Call stack and then run execute the first operation.</p>
<p>if the operation is <strong>synchronous</strong> example like arithmetique operation it will be executed imediatly in the main thread and then getting to the next operation.</p>
<p>if it's <strong>Asynchronous</strong> operation like disc or network operation , Nodejs will not execute it in the main thread as it will block it but it will pass it to the LibUV and continue the execution to the next operation.</p>
<p>LibUV will launch the operation By using a thread worker, child process or OS handler and queued it's callback in the correspending Event Queue.</p>
<p>the Event loop will ensure the all the syncronous code will be executed first , then if the call stack is empty  it will check for  completed operations by invoking the registered callback functions.</p>
<p>when concurrent IO operation are ready to be executed , the event loop manage the priority between the callbacks</p>
<h2>The Event loop Phases</h2>
<p>as we disccus earlied the event loop is the piece of code responsible for handling asyncronous operation.it's activly checking Event queues.in the case of concurency the event loop follow this order of priority.</p>
<p>First, there is the <strong>timer queue</strong> , which holds callbacks associated with setTimeout and setInterval.</p>
<p>Second, there is the <strong>I/O queue</strong> which contains callbacks associated with all the async methods such as methods associated with the fs and http modules.</p>
<p>Third, there is the <strong>check queue</strong> which holds callbacks associated with the setImmediate function, which is specific to Node.</p>
<p>Fourth, there is the <strong>close queue</strong> which holds callbacks associated with the close event of an async task.</p>
<p>Finally, there is the <strong>microtask queue</strong> which contains two separate queues.</p>
<ul>
<li><em>nextTick</em> queue which holds callbacks associated with the process.nextTick function.</li>
<li><em>Promise</em> queue which holds callbacks associated with the native Promise in JavaScript.</li>
</ul>
<h2>What to keep in Mind when developing Node js application</h2>
<p>here is some points that i throught about regarding the event loop and asynchronous operation</p>
<h3>Blocking the Event Loop:</h3>
<ul>
<li>
<p>always use Asynchronous API whenever it's possible : for example FS module that enable file operations on the OS provide both synchronous and asynchronous API,my go to is the async API unless you really knowing what you are doing</p>
</li>
<li>
<p>Avoid  Performing Intensive CPU computing : as this will block other  asynchronous operations  ,it's better to overload it to another thread worker or spinning a child process to handle the computing.</p>
</li>
</ul>
<h3>Inefficient Resource Utilization:</h3>
<p>perform intensive CPU computing will consume excessive CPU resources,leading to poor resource utilization. Instead, breaking the task into smaller.</p>
<h3>Inadequate Concurrency Management:</h3>
<p>Proper utilization of techniques like promises, async/await, and callback patterns can help manage concurrency effectively and avoid issues like race conditions or deadlocks. you have to Understand the priority and the order of execution of your asynchronous code  is crutial espicially that node js is an event driven runtime.</p>
<h3>Lack of Error Handling:</h3>
<p>use error handling techniques such as try-catch blocks, error events, or promises' rejection handling , otherwise you'll end up with  unhandled exceptions or silent failures that can be hard to trace.</p>
<h3>Learn about the Best Practices in generale is required:</h3>
<p>you have to learn when to use various technics that was designed to solve certain problem like caching technique, streams -based API ,clusting and so On</p>
<h2>Further Resources</h2>
<p>i have found this material helpful to learn more about the topic and i want to share it with you:</p>
<ul>
<li>
<p>Everything You Need to Know About Node.js Event Loop - Bert Belder, IBM on <a href="https://www.youtube.com/watch?v=PNa9OMajw9w">youtube</a></p>
</li>
<li>
<p>Node's Event Loop From the Inside Out by Sam Roberts, IBM on <a href="https://www.youtube.com/watch?v=P9csgxBgaZ8">youtube</a></p>
</li>
<li>
<p>Nodejs | Event loop,timer and Next tick  <a href="https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick">Official Docs</a></p>
</li>
<li>
<p>What the heck is the event loop anyway | Philip Roberts | JSConf EU on <a href="https://www.youtube.com/watch?v=8aGhZQkoFbQ">youtube</a></p>
</li>
<li>
<p>Here's Why Node JS is NOT Single Threaded on <a href="https://youtu.be/JwSEqEalg7E">youtube</a></p>
</li>
<li>
<p>The Node.js Event Loop: Not So Single Threaded on <a href="https://www.youtube.com/watch?v=zphcsoSJMvM">youtube</a></p>
</li>
<li>
<p>Don't Let Just Node.js Take the Blame! - Danial Khan on <a href="https://youtu.be/r1omVx3Phrw">youtube</a></p>
</li>
<li>
<p>Event loop in JavaScript on <a href="https://acemood.github.io/2016/02/01/event-loop-in-javascript/">AceMood's Blog</a></p>
</li>
<li>
<p>Event Loop and the Big Picture — NodeJS Event Loop  By Deepal Jayasekara
<a href="https://blog.insiderattack.net/event-loop-and-the-big-picture-nodejs-event-loop-part-1-1cb67a182810">Meduim</a></p>
</li>
<li>
<p>LibUV Design <a href="http://docs.libuv.org/en/v1.x/design.html#handles-and-requests"> LibUV official documentation</a></p>
</li>
<li>
<p>using libuv and http-parser to build a webserver <a href="https://vimeo.com/24713213">vimeo</a></p>
</li>
</ul>

Nodejs Event Loop and Async I/O (Overview)

<img src="/assets/introduction-nest-js-nodejs-framework/cover.png" alt="cover image">
<h2>Introduction;</h2>
<p>this post will be deleted in no time , just be patient</p>
<h2>What compose Nodejs environment</h2>
<h2>The execution flow of execution :</h2>
<h2>The Event loop Phases</h2>

What makes Nestjs the Go to Nodejs Framework

<img src="/assets/file-storage-git-versioning/cover.png" alt="cover image for version control file storage">
<h2>TLDR;</h2>
<p>in this blog post , we will build a File editor with <strong>version control features</strong> like saving a changes , logging the history of changes , track who made a change ,when and More .</p>
<p>we like Typescipt , we like the Web , so we will use Js frameworks in our development , React Js for the UI and NodeJs for our Backend application that will use File system API and Git .</p>
<h2>Introduce Git : the Version control system (VCS)</h2>
<p>the term 'version control' may be new to some of us. in the contexte of this blog we are refering to version control systems. which is a type of software that enable teams of software developers to collaborate in writing into one source code, by providing features like tracking the history of changes that occures over time. who and when a changes was introduced , rollback to certain version. branching and more .</p>
<p>Git stand as the most popular VCS by far in the recent years due to it's many adventages including it's distributed approche.
each collaborator can clone a local version of the central server without relaying on it's  availability online . and  can  synchronize with the central server whether it's by pulling or pushing new changes. a local version has all the history of changes and banches.</p>
<img src="/assets/file-storage-git-versioning/git.png" alt="git distributed veersion control system">
<p>For the sake of simplicity and to mantain the scope of this blog we will not deep dive into Git and just introduce some basic terms.</p>
<p><strong>repository</strong> is a collection of files and their complete history of changes.</p>
<p><strong>branch</strong> is a copy of a repository that allows developers to work on new features or bug fixes without affecting the main codebase.</p>
<p><strong>Commits</strong> are changes made to a file or a group of files, and each commit is associated with a unique identifier called a hash.</p>
<p><strong>Merging</strong> is the process of combining changes from different branches into one.</p>
<p>here we present the most used  Git commands</p>
<ul>
<li><strong>Pull:</strong>  used to fetch and integrate changes from a remote repository into a local one.</li>
<li><strong>Push:</strong>  used to upload local changes to a remote repository.</li>
<li><strong>Add:</strong>   used to add new or modified files to the Git index (also known as staging), preparing them to be committed.</li>
<li><strong>Commit:</strong> used to record changes to the repository. It creates a new revision in the repository's history that represents the changes made to files in the repository. Each commit is identified by a unique hash code and includes a commit message that describes the changes made.</li>
<li><strong>Log:</strong>  used to view the history of a repository, including all the commits that have been made, who made them, and when they were made. The log command can be used to view detailed information about each commit, such as the commit message, author, and changes made.</li>
<li><strong>Revert:</strong>  used to undo changes made in a specific commit or range of commits, effectively reversing their effects.</li>
<li><strong>Show:</strong> used to display the details of a specific commit or the changes made to files in the repository. It is used to inspect the content and metadata . and can be useful for reviewing changes or debugging issues in a repository.</li>
</ul>
<p>keep in mind that these git commands comes with some powerful sets of parameters that customize the behaviour of the commands,for example here is what we can use with <strong>Git log</strong></p>
<p><strong>--grep</strong>: searches for commits with a specific message or commit message pattern</p>
<p><strong>--since/--after and --until/--before:</strong> filters commits by date or date range</p>
<p><strong>--author:</strong> filters commits by author</p>
<p><strong>-- $pathname:</strong> to query only the commit of specifique path directory or file.</p>
<h2>Why Mentionning Git , Well .</h2>
<p>you may say that the goal is to build a file editor with version control features but why mentionning Git , if you thought that we gonna build the same features as git or at least some of them the answer is you're wrong . and if you thought that we gonna use Git to help us in the process of development and collaborating you're wrong also . we mention Git because we gonna integrate it with our Nodejs backend application and benifite from it's powerfull commands to control the changes of the files that we will manipulate with our Nodejs File system API (we will explore it in the next section).</p>
<p>While we could interact with Git by  running a child process to call the git command-line program .
we prefere to use a dedicated library that facilitate the interface .a library like “simple-git”  enhance our development experience by providing the simpilcity of usage ,the types safety and more .</p>
<p>here is some code samples using  "simple-git" library.</p>
<h3>initialise the repository</h3>
<pre><code>const git : SimpleGit= simpleGit(BASE_DIR, { binary: 'git' });
await git.init()
</code></pre>
<h3>add some metadata</h3>
<pre><code>await git.addConfig('user.name', 'ayoub bouguettaya');
await git.addConfig('user.email', `author@foobarz.blog`);
</code></pre>
<h3>stage the changes</h3>
<pre><code>await git.add('.');
</code></pre>
<h3>commit the staged changes</h3>
<pre><code>await git.commit("first commit");
</code></pre>
<h3>push the changes to the  remote repository</h3>
<pre><code>await git.push("origin","main","--force")
</code></pre>
<h2>Let's cook</h2>
<p>we said that it's a file editor but no it's not that great idea , but we don't have too many options , we implemente a workspace</p>
<p>implemente a tree ;  if someone is writing to a file , it's considered locked until it get realised .</p>
<h2>File Editor</h2>
<p>Node.js provides a File System module called 'fs', which allows us to communicate with the operating system's file system and perform all possible operations on files and directories.</p>
<p>The module provides both synchronous and asynchronous API( Promises based API was added in version control, in addition to the callback based API). The Promise-based APIs are preferred because they can make it easier to write asynchronous code that is more readable and less error-prone than callback-based code.</p>
<p>Please check  <a href="https://nodejs.org/api/fs.html#promises-api">the official nodejs documentation</a> for more information.
Below are some code samples,that illustrate how it's simple to create directory ,check file status ,read,write in files ect.</p>
<h3>Import FS Promises-based methods</h3>
<pre><code class="language-{javascript}">import { readFile, writeFile, mkdir,  stat ,access} from 'node:fs/promises';
</code></pre>
<h3>checking if the file exists .if it can't access it will throw an error.</h3>
<pre><code class="language-{javascript}">await access(fullPath);
</code></pre>
<h3>Read File</h3>
<pre><code class="language-{javascript}">console.log(await readFile(fullPath, { encoding: 'utf8' })   )
</code></pre>
<h3>write in file / overwrite of the content</h3>
<pre><code class="language-{javascript}">await writeFile(fullPath, newContent)
</code></pre>
<h3>create directory</h3>
<pre><code class="language-{javascript}">await mkdir(directory, { recursive: true })
</code></pre>
<h1>Let's Play with code</h1>
<p>After introducing the concepts .we are going to build a small application to explore control versionning of a file  in NodeJs. this application could be a starter application for ready production content management or document editor application that need’s collaboration and versioning features.</p>
<h2>overview</h2>
<img src="/assets/file-storage-git-versioning/overview.png" alt="design system overview">
<p><strong>backend:</strong>  this is the nodejs application that will handle the operation ,it will basically provide REST API .</p>
<p><strong>frontend:</strong> this will be the ReactJs application that provide our user interface.</p>
<p><strong>Git:</strong> the tool git need to installed in our machine.</p>
<p><strong>file storage:</strong> we should have right access to read / write on file storage inside our machine .</p>
<p><strong>Browser:</strong> this is where we will be playing.</p>
<h2>The Backend Part</h2>
<p>We will be using NestJs, which is a Node.js framework, and getting started with Nest CLI to scaffold the project and generate some boilerplate code</p>
<ul>
<li>controller class to define REST API Routing.</li>
<li>service class to handle the Requests and Logics</li>
</ul>
<h3>Design our  Rest API Endpoints</h3>
<ul>
<li>
<p><strong>GET /data</strong>: the endpoint to get the Data in YAML format .by reading the file "setting.yaml" in the repos</p>
</li>
<li>
<p><strong>POST /data</strong>: endpoint to save the data using File system Api 'writeFile' to the file path after that using the Git Commands in this order. <code>git add .</code> to stage the changes of the file , <code>git commit</code>  to record the changes staged for the file along with some meta data like the <strong>author</strong> ,<strong>comment</strong> and <strong>timestamps</strong> .</p>
</li>
</ul>
<pre><code>  async saveData(saveDataDto: SaveDataDto) {
    const { data, commitMessage, author } = saveDataDto;

    await writeFile(getFilePath(), data);
    await this.git.addConfig('user.name', author || 'unkown');
    await this.git.addConfig('user.email', `${author}@foobarz.blog`);

    await this.git.add('.');
    await this.git.commit(commitMessage);

    return data;
  }
</code></pre>
<ul>
<li><strong>GET /commit-history</strong>: endpoint to get the list of git commit history of the file with pagination</li>
</ul>
<pre><code>async getCommitHistory(pagination: GetHistoryPaginationDto) {
    const {
      size = 10,
      page = 1,
    } = pagination;

    const query: string[] = []; // we can combine multiple queries like grep ,since ,after,author ect 
    const fileName = getFilePath()

    const { total: totalCount } = await this.git.log&#x3C;DefaultLogFields>([
      ...query,
      '--',
      fileName,
    ]);

    const { all: commits, total: pageLength } =
      await this.git.log&#x3C;DefaultLogFields>([
        `--max-count=${size}`,
        `--skip=${(page - 1) * size}`,
        ...query,
        '--',
        fileName,
      ]);

    const maxPages = Math.ceil(totalCount / size);

    return {
      items: commits,
      pageLength: pageLength,
      page: page,
      maxPages: maxPages,
      size: size,
      totalCount,
    };
  }
</code></pre>
<ul>
<li><strong>GET /data/:commitHash</strong>: the endpoint to get the  file,changes,author and the comment linked to a commit hash.</li>
</ul>
<pre><code>
  async getChangesAndDataStateByCommit(commitHash: string) {
    try {
      const pathfileName = getFilePath();

      const data = await this.git.show([`${commitHash}:${FileName}`]);
      const changes = await this.git.show([commitHash, '--pretty=format:%b', '--', pathfileName]);

    const { all: commits } =
    await this.git.log&#x3C;DefaultLogFields>([
      `--max-count=1`,
      commitHash,
      '--',
      pathfileName,
    ]);

      const author = commits[0].author_name
      const subject = commits[0].message

      return { data, changes ,author,subject}
    } catch (error) {
      throw new Error(error.message);
    }
  }
</code></pre>
<h2>The Frontend Part</h2>
<p>Scaffold our react application with   with Vite Js
<code>yarn create vite frontend-app --template react-ts</code>
then adding "react-router-dom" for our routing  and create two pages</p>
<ul>
<li>The Home Page on  <code>/</code></li>
<li>The Commit changes Page on <code>/show/:commitHash</code></li>
</ul>
<p>the HomePage is composed from 3 main components:</p>
<ul>
<li>
<p>AuthenticatedUserPanel: this component is displaying the current authenticated user as a string and a form that allow us to change the authenticated user (which is just mocking move we are not handling any real authentication)</p>
</li>
<li>
<p>HistoricPanel: this component display the history of commits , eash commit has an author ,a message, timestamps and a unique Hash. for each commit item there is a possibiliy to show the details of the commit by ppresing on the "show" button which will redirect the user to <code>/show/:hash-commit</code></p>
</li>
<li>
<p>Editor Panel: this component allow us to edit the content of the file in a sofisticated matter taking into consideration YAML format.thanks to '@monaco-editor/react' for the awesome Editor.</p>
</li>
</ul>
<p>the Commit changes Page  <code>/show/:commitHash</code> is composed with one main component:</p>
<p>using this library 'react-diff-view' we had easly able to display a nice looking Diff view to show the changes that was introduce by a given commit.</p>
<hr>
<p>We reached The End 😎  hopefully you have found the post useful in someway. Now you can  <a href="https://github.com/ayoubbouguettaya/version-control-file-storage/archive/refs/heads/main.zip">clone the project</a> ,view the source code, make some changes as you explore the version control capalities with git  ,run it in your machine and have fun with the application 🚀.</p>
<blockquote>
<p>All the illustation used in this blog are my personel sketch on excalidraw.com, my references on system version control were essentially the officiel website of git , atlassian.com / bitbuckets  blog ,and some awesome meduim / stackoverflow posts where i get to know the npm packages "simple-git" ,"react-diff-view" and "@monaco-editor/react"</p>
</blockquote>

Nodejs Event Loop and Async I/O (Overview)

What makes Nestjs the Go to Nodejs Framework

Build a File Editor with version control features...