Sometimes your application will provide so little data, that it may be overkill to provide an API to access it. In such case you might want to embedded the database within your application. Actually, this technique is used for a while in mobile applications. It can be useful to speed up low bandwidth devices, boost application launch and more. Thing is, I’m a web developer and I wanted to be able to do the same in one of my web application.
The recommended and compatible way to store data in Browser is by using the IndexedDB. Thing is, it uses a different approach than pure SQL, as explained in documentation. The API is also tedious to use, that is why MDN recommend to use more programmer-friendly library for simple usages. And to finish, IndexedDB format being younger it is not as supported as SQLite.
Based on that, I wanted to try to load a SQLite database into my browser and request it. The database contains some 13 thousand entries about file extension information. The application provides the company and software associated to a file extension searched by users.
MDN describes Web Assembly as a way to run code written in multiple languages on the web at near native speed, allowing us to run a whole new class of programs. It is a low-level assembly-like language with a compact binary format that runs with near-native performance and provides languages such as C/C++ and Rust with a compilation target so that they can run on the web. It works by compiling code to LLVM byte-code and loading it using Emscripten.
MDN将Web Assembly描述为一种以接近本机的速度运行以多种语言编写的代码的方法,从而使我们能够运行全新的程序类。 它是一种类似于汇编语言的低级语言,具有紧凑的二进制格式,具有接近本机的性能,并为C / C ++和Rust等语言提供了编译目标,以便它们可以在Web上运行。 它通过将代码编译为LLVM字节代码并使用Emscripten加载来工作。
Web Assembly网络组装
Usages can be : * Porting a C/C++ application with Emscripten. * Writing or generating Web Assembly directly at the assembly level. * Writing a Rust application and targeting Web Assembly as its output. * Using AssemblyScript which looks similar to Type Script and compiles to Web Assembly binary.
用法可以是:*用Emscripten移植C / C ++应用程序。 *直接在程序集级别编写或生成Web程序集。 *编写Rust应用程序,并以Web Assembly作为输出。 *使用外观类似于Type Script的AssemblyScript并编译为Web Assembly二进制文件。
SQL.js (SQL.js)
A while ago I’ve heard about a project to load and parse PDF files directly in the browser and learned that it uses a port of PDFium. So I begun to search for a port of SQLite client and by chance, found the SQL.js awesome project done by Alon Zakai. It provides a port to Web Assembly and a JavaScript loader.
To install, it was a breeze as there is a first release and an npmjs package. You can check the documentation for different usages.
要安装,很容易,因为有第一个版本和npmjs软件包。 您可以查看文档中的不同用法。
Webpack (Webpack)
So I decided to start working on this project, because I didn’t find any API listing file extensions. I made a SQLite database and created a React application to showcase it.
While in the process of implementing the solution I found some blockers that I wanted to share for those having the same issues.
在实施解决方案的过程中,我发现了一些我想与遇到同样问题的人分享的障碍。
To run SQL.js the module need to load the WASM file in order to parse the SQLite database. Thing is, because I used Webpack 4, I always got the following error when trying to load it from the bundle :
`Web Assembly module is included in initial chunk. This is not allowed, because Web Assembly download and compilation must happen asynchronous.`
Web程序集模块包含在初始块中。 不允许这样做,因为Web程序集的下载和编译必须异步进行。
Making some researches learned me that Webpack is not fully compatible with Web Assembly for the moment, so I couldn’t include it with my bundle. So I hacked my way by loading directly the wasm file from SQL.js repository (not a big fan of that but it worked) :
let config = { locateFile: filename =>`https://cdn.rawgit.com/kripken/sql.js/v1.0.1/dist/${filename}`}
This way you can pass the `config` object to SQL.js initialization function and it will load the wasm from the specified location.
这样,您可以将config对象传递给SQL.js初始化函数,它将从指定位置加载wasm。
打包数据库 (Pack the database)
I also needed to embedded the database with my application, this was more straightforward as Webpack provides a file-loader to include any binary file to your project. To use it you just have to add this to your Webpack configuration file.
After that, you just import your database to get the URL on your server. I did it this way :
之后,只需导入数据库即可在服务器上获取URL。 我这样做是这样的:
import database from'../assets/database.sqlite';
加载数据库 (Load the database)
Now that we have our SQL.js initialized and our database packed with our application we need to load it as SQL.js do not provide database loading by URL. The documentation shows you how to do it, I used the same code and specified the database path.
var xhr = newXMLHttpRequest();xhr.open('GET', database, true);xhr.responseType = 'arraybuffer';xhr.onload = () => { var uInt8Array = newUint8Array(xhr.response); var db = newSQL.Database(uInt8Array);};xhr.send();
请求数据库 (Request the database)
To finish, I added a simple query in the application to allow users to search by file extension using the`db.exec` function provided by SQL.js.
You can check the relevant code in the `Search.js` React component in the project. It will search based on requested extension and update the Redux store afterwards.
Hoping that it will be useful for those who wants to embedded little databases within their web browser applications. Moreover I was thinking that by not accessing an API to request the database on a server, this removes server security breach through SQL injection.