我编写了一个命令行实用程序,该实用程序生成大量数据,然后将生成的数据转储到一个大文件中。
我注意到一些竞争条件问题,其中一个进程读取文件,并在另一个文件正在读取/写入相同内容的同时将其添加到文件中。这会导致某些数据从最终文件中删除/覆盖,因为当一个进程中添加的内容合并生成的内容时,该数据不会读入另一个进程中。
我目前正在研究使用NodeJs互斥库来尝试向进程添加一些读/写锁。我还可以尝试将子进程重构为单进程异步任务,但这会大大降低操作速度,并且需要大量额外工作。
async-mutexlockfile
async-mutex
const definition = await generateData();
await mutex.runExclusive(() => {
console.log('LOCK ACQUIRED FOR', uuid);
if (fs.existsSync(baseFile)) {
baseDefinition = yaml.safeLoad(fs.readFileSync(baseFile, 'utf8'));
}
baseDefinition = _.merge(baseDefinition, definition);
fs.writeFileSync(baseFile, yaml.safeDump(baseDefinition));
console.log('RELEASING LOCK FOR', uuid);
});
const moreData = await generateMoreData();
await mutex.runExclusive(async () => {
console.log('SECOND LOCK ACQUIRED FOR', uuid);
await moveData(moreData, baseFile);
console.log('SECOND LOCK RELEASED FOR', uuid);
});
generateDatagenerateMoreDatamoveDatabaseFile
运行此命令时,我得到一些如下所示的输出
LOCK ACQUIRED FOR 526ac716-b135-4a2d-986d-75bee856411b
LOCK ACQUIRED FOR 104073b7-6d7f-4ca1-8513-eec103877898
RELEASING LOCK FOR 526ac716-b135-4a2d-986d-75bee856411b
SECOND LOCK ACQUIRED FOR 526ac716-b135-4a2d-986d-75bee856411b
RELEASING LOCK FOR 104073b7-6d7f-4ca1-8513-eec103877898
SECOND LOCK ACQUIRED FOR 104073b7-6d7f-4ca1-8513-eec103877898
SECOND LOCK RELEASED FOR 526ac716-b135-4a2d-986d-75bee856411b
LOCK ACQUIRED FOR 155eb578-bc91-4bee-830f-bab2e1363bb8
SECOND LOCK RELEASED FOR 104073b7-6d7f-4ca1-8513-eec103877898
LOCK ACQUIRED FOR 8e6e08cf-8b3b-4040-af88-86670ad5cbb1
RELEASING LOCK FOR 155eb578-bc91-4bee-830f-bab2e1363bb8
SECOND LOCK ACQUIRED FOR 155eb578-bc91-4bee-830f-bab2e1363bb8
LOCK ACQUIRED FOR cb79f948-e4a4-4f75-ba31-fa2b743ce886
RELEASING LOCK FOR 8e6e08cf-8b3b-4040-af88-86670ad5cbb1
SECOND LOCK ACQUIRED FOR 8e6e08cf-8b3b-4040-af88-86670ad5cbb1
SECOND LOCK RELEASED FOR 155eb578-bc91-4bee-830f-bab2e1363bb8
RELEASING LOCK FOR cb79f948-e4a4-4f75-ba31-fa2b743ce886
SECOND LOCK ACQUIRED FOR cb79f948-e4a4-4f75-ba31-fa2b743ce886
SECOND LOCK RELEASED FOR 8e6e08cf-8b3b-4040-af88-86670ad5cbb1
SECOND LOCK RELEASED FOR cb79f948-e4a4-4f75-ba31-fa2b743ce886
如您所见,由于锁的获取和释放是彼此独立的,因此锁显然不会在进程之间共享。
NodeJ中是否有一种方法可以独立于文件的运行进程来锁定文件?