Skip to content

Instantly share code, notes, and snippets.

@lmammino

lmammino/README.md

Last active June 27, 2022 19:30
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save lmammino/3743adf55c08f0b7d7bd229b128b37ec to your computer and use it in GitHub Desktop.
Save lmammino/3743adf55c08f0b7d7bd229b128b37ec to your computer and use it in GitHub Desktop.
Download ZIP
Request batching benchmark
Raw
README.md

Request batching benchmark

Request batching is a Node.js pattern that can be used to optimise how a web server handles identical concurrent requests.

It allows to process the request only once for all the clients concurrently requesting the same information. This can save expensive round trips to backend services and can avoid backend services to be overloaded with many concurrent identical requests. In a way it's a more specialised version of micro-caching.

This can lead to significant performance improvement in cases where there are many concurrent users requesting the same page.

The implementation here exploits the use of promises to track pendind requests. If the same request is made while there is already one pending, the same pending promise will be awaited for the new request, which saves additional requests to backend services.

Once the promise resolves, all the client awaiting a responde will receive it.

To run the benchmarks you can use autocannon:

npx autocannon -c 20 -d 20 --on-port /api/hotels/rome -- node server.mjs

To compare the results with a version that does not use request batching just comment line 19 and re-run the command above.

ℹ️ Note This is an artificial benchmark and results might vary significantly in real-life scenario. Always run your own benchmarks before deciding whether this optimization can have a positive effect for you.

Raw
server.mjs
import { createServer } from 'http'
const mockHotels = [
{ name: 'the house of zeus' },
{ name: 'wild rooster motel' },
{ name: 'zagreus palace' },
]
let numConcurrent = new Map()
let pendingRequests = new Map()
const db = {
query (q) {
return new Promise((resolve, reject) => {
const [cityId] = q.values
// simulates the db getting progressively slower based
// on the number of concurrent queries
const responseTime = Math.random() * numConcurrent.get(cityId)
setTimeout(() => resolve(mockHotels), responseTime)
})
}
}
function getHotelsForCity (cityId) {
if (pendingRequests.has(cityId)) {
let concurrent = numConcurrent.get(cityId) || 0
numConcurrent.set(cityId, concurrent + 1)
// *** comment the following line to disable request batching ***
return pendingRequests.get(cityId)
}
const asyncOperation = db.query({
text: 'SELECT * FROM hotels WHERE cityid = $1',
values: [cityId],
})
pendingRequests.set(cityId, asyncOperation)
asyncOperation.finally(() => {
pendingRequests.delete(cityId)
numConcurrent.delete(cityId)
})
return asyncOperation
}
const urlRegex = /^\/api\/hotels\/([\w-]+)$/
const server = createServer(async (req, res) => {
const url = new URL(req.url, 'http://localhost')
const matches = urlRegex.exec(url.pathname)
if (!matches) {
res.writeHead(404, 'Not found')
return res.end()
}
const [_, city] = matches
const hotels = await getHotelsForCity(city)
res.writeHead(200)
res.end(JSON.stringify({ hotels }))
})
server.listen(8000)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment