feat(MongoDumpTarget): Implement core MongoDumpTarget methods and update documentation & project configs
This commit is contained in:
261
readme.md
261
readme.md
@@ -1,76 +1,259 @@
|
||||
# @push.rocks/mongodump
|
||||
a tool to handle dumps of MongoDB databases
|
||||
|
||||
## Install
|
||||
To use @push.rocks/mongodump in your project, run:
|
||||
**A powerful MongoDB backup and restore tool** 🚀
|
||||
|
||||
[](https://www.npmjs.com/package/@push.rocks/mongodump)
|
||||
[](https://www.npmjs.com/package/@push.rocks/mongodump)
|
||||
[](https://www.typescriptlang.org/)
|
||||
[](https://nodejs.org/api/esm.html)
|
||||
|
||||
## What it does 🎯
|
||||
|
||||
`@push.rocks/mongodump` is your go-to solution for creating and managing MongoDB database dumps. Whether you're backing up critical production data, migrating between environments, or implementing disaster recovery strategies, this tool has got your back. It provides a clean, TypeScript-based API for:
|
||||
|
||||
- 📦 **Full database backups** - Export entire MongoDB databases
|
||||
- 🗂️ **Collection-level dumps** - Selectively backup specific collections
|
||||
- 🏷️ **Custom naming** - Transform document names during export
|
||||
- 🗜️ **Archive support** - Create compressed tar archives of your dumps
|
||||
- 🔄 **Async/await patterns** - Modern promise-based workflow
|
||||
- 🎨 **TypeScript first** - Full type safety and IntelliSense support
|
||||
|
||||
## Installation 💻
|
||||
|
||||
```bash
|
||||
# Using npm
|
||||
npm install @push.rocks/mongodump --save
|
||||
|
||||
# Using pnpm (recommended)
|
||||
pnpm add @push.rocks/mongodump
|
||||
|
||||
# Using yarn
|
||||
yarn add @push.rocks/mongodump
|
||||
```
|
||||
|
||||
## Usage
|
||||
## Quick Start 🚀
|
||||
|
||||
This guide provides an overview of how to work with @push.rocks/mongodump to handle dumps of MongoDB databases efficiently.
|
||||
Get up and running in seconds:
|
||||
|
||||
### Setting up a MongoDB Dump Target
|
||||
First, you'll need to describe your MongoDB database using an interface provided by the module. Here's a sample descriptor:
|
||||
```typescript
|
||||
import { MongoDump } from '@push.rocks/mongodump';
|
||||
import type { IMongoDescriptor } from '@tsclass/tsclass';
|
||||
|
||||
// Define your MongoDB connection
|
||||
const mongoDescriptor: IMongoDescriptor = {
|
||||
mongoDbName: 'my-database',
|
||||
mongoDbUser: 'admin',
|
||||
mongoDbPass: 'secure-password',
|
||||
mongoDbUrl: 'mongodb+srv://cluster.mongodb.net',
|
||||
};
|
||||
|
||||
// Create a dump of your entire database
|
||||
const mongoDump = new MongoDump();
|
||||
const target = await mongoDump.addMongoTargetByMongoDescriptor(mongoDescriptor);
|
||||
await target.dumpAllCollectionsToDir('./backups/my-backup');
|
||||
|
||||
// Done! Your backup is ready 🎉
|
||||
await mongoDump.stop();
|
||||
```
|
||||
|
||||
## Detailed Usage 📚
|
||||
|
||||
### Setting Up MongoDB Connection
|
||||
|
||||
The module uses MongoDB descriptors to establish connections. This approach provides flexibility and security:
|
||||
|
||||
```typescript
|
||||
import { IMongoDescriptor } from '@tsclass/tsclass';
|
||||
import { MongoDump, MongoDumpTarget } from '@push.rocks/mongodump';
|
||||
|
||||
const myMongoDescriptor: IMongoDescriptor = {
|
||||
mongoDbName: '<database_name>',
|
||||
mongoDbUser: '<database_user>',
|
||||
mongoDbPass: '<database_password>',
|
||||
mongoDbUrl: 'mongodb+srv://<user>:<password>@<cluster_url>/<dbname>?retryWrites=true&w=majority',
|
||||
const mongoDescriptor: IMongoDescriptor = {
|
||||
mongoDbName: 'production_db',
|
||||
mongoDbUser: process.env.MONGO_USER,
|
||||
mongoDbPass: process.env.MONGO_PASS,
|
||||
mongoDbUrl: process.env.MONGO_URL,
|
||||
};
|
||||
```
|
||||
|
||||
### Creating and Using the MongoDump Instance
|
||||
To interact with MongoDB for dumping purposes, you'll use the `MongoDump` class:
|
||||
### Working with MongoDump
|
||||
|
||||
The `MongoDump` class is your main entry point for managing database dumps:
|
||||
|
||||
```typescript
|
||||
async function setupMongoDump() {
|
||||
const mongoDump = new MongoDump();
|
||||
const mongoDumpTarget = await mongoDump.addMongoTargetByMongoDescriptor(myMongoDescriptor);
|
||||
const mongoDump = new MongoDump();
|
||||
|
||||
// mongoDumpTarget can now be used for further operations
|
||||
}
|
||||
setupMongoDump();
|
||||
// Add a MongoDB target
|
||||
const dumpTarget = await mongoDump.addMongoTargetByMongoDescriptor(mongoDescriptor);
|
||||
|
||||
// The target is now ready for dump operations
|
||||
```
|
||||
|
||||
### Dumping Collections to a Directory
|
||||
To dump the collections of a database into a directory with files representing each document, you can use the `dumpCollectionToDir` method. This method is useful for creating backups or migrating data:
|
||||
### Dumping Collections
|
||||
|
||||
#### Dump All Collections to Directory
|
||||
|
||||
Perfect for complete database backups:
|
||||
|
||||
```typescript
|
||||
async function dumpCollections() {
|
||||
const mongoDump = new MongoDump();
|
||||
const mongoDumpTarget = await mongoDump.addMongoTargetByMongoDescriptor(myMongoDescriptor);
|
||||
// Basic dump - uses document _id as filename
|
||||
await dumpTarget.dumpAllCollectionsToDir('./backups/full-backup');
|
||||
|
||||
await mongoDumpTarget.dumpAllCollectionsToDir('./path/to/dumpDir', null, true);
|
||||
// This dumps all collections to the specified directory, cleaning the directory before dumping.
|
||||
}
|
||||
dumpCollections();
|
||||
// Custom naming - use any document field for filenames
|
||||
await dumpTarget.dumpAllCollectionsToDir(
|
||||
'./backups/named-backup',
|
||||
(doc) => `${doc.username}_${doc.timestamp}`,
|
||||
true // Clean directory before dumping
|
||||
);
|
||||
```
|
||||
|
||||
### Advanced Dumping Options
|
||||
For more control over the dumping process, including naming conventions for dumped files or handling specific collections, you can explore methods like `dumpCollectionToDir` for individual collections and advanced configurations concerning directory cleanliness and document naming.
|
||||
#### Dump Specific Collection
|
||||
|
||||
### Shutting Down Properly
|
||||
It's important to close down database connections properly once your dumping operations are complete:
|
||||
When you need granular control:
|
||||
|
||||
```typescript
|
||||
async function shutDownMongoDump() {
|
||||
// Get available collections
|
||||
const collections = await dumpTarget.getCollections();
|
||||
console.log('Available collections:', collections);
|
||||
|
||||
// Dump a specific collection
|
||||
const userCollection = collections.find(c => c.collectionName === 'users');
|
||||
await dumpTarget.dumpCollectionToDir(
|
||||
userCollection,
|
||||
'./backups/users',
|
||||
(doc) => doc.email.replace('@', '_at_') // Custom naming
|
||||
);
|
||||
```
|
||||
|
||||
### Archive Support
|
||||
|
||||
Create compressed archives for efficient storage and transfer:
|
||||
|
||||
```typescript
|
||||
// Dump to tar archive file
|
||||
await dumpTarget.dumpCollectionToTarArchiveFile(
|
||||
userCollection,
|
||||
'./backups/users.tar.gz'
|
||||
);
|
||||
|
||||
// Or work with streams for advanced scenarios
|
||||
const archiveStream = await dumpTarget.dumpAllCollectionsToTarArchiveStream();
|
||||
// Pipe to S3, network storage, etc.
|
||||
```
|
||||
|
||||
### Advanced Patterns
|
||||
|
||||
#### Scheduled Backups
|
||||
|
||||
Implement automated backup strategies:
|
||||
|
||||
```typescript
|
||||
import { CronJob } from 'cron';
|
||||
|
||||
const backupJob = new CronJob('0 0 * * *', async () => {
|
||||
const timestamp = new Date().toISOString().split('T')[0];
|
||||
const mongoDump = new MongoDump();
|
||||
const target = await mongoDump.addMongoTargetByMongoDescriptor(mongoDescriptor);
|
||||
|
||||
await target.dumpAllCollectionsToDir(
|
||||
`./backups/daily/${timestamp}`,
|
||||
null,
|
||||
true
|
||||
);
|
||||
|
||||
await mongoDump.stop();
|
||||
// Closes all open database connections gracefully
|
||||
}
|
||||
shutDownMongoDump();
|
||||
console.log(`✅ Daily backup completed: ${timestamp}`);
|
||||
});
|
||||
|
||||
backupJob.start();
|
||||
```
|
||||
|
||||
### Conclusion
|
||||
The @push.rocks/mongodump module provides a flexible approach to handling MongoDB database dumps, whether it's for backup, migration, or other purposes. By leveraging TypeScript and modern async patterns, it integrates smoothly into modern Node.js applications focused on MongoDB interactions.
|
||||
#### Multiple Database Targets
|
||||
|
||||
Handle multiple databases simultaneously:
|
||||
|
||||
```typescript
|
||||
const mongoDump = new MongoDump();
|
||||
|
||||
// Add multiple targets
|
||||
const prodTarget = await mongoDump.addMongoTargetByMongoDescriptor(prodDescriptor);
|
||||
const stagingTarget = await mongoDump.addMongoTargetByMongoDescriptor(stagingDescriptor);
|
||||
|
||||
// Dump both in parallel
|
||||
await Promise.all([
|
||||
prodTarget.dumpAllCollectionsToDir('./backups/production'),
|
||||
stagingTarget.dumpAllCollectionsToDir('./backups/staging')
|
||||
]);
|
||||
|
||||
await mongoDump.stop();
|
||||
```
|
||||
|
||||
#### Error Handling
|
||||
|
||||
Implement robust error handling:
|
||||
|
||||
```typescript
|
||||
try {
|
||||
const mongoDump = new MongoDump();
|
||||
const target = await mongoDump.addMongoTargetByMongoDescriptor(mongoDescriptor);
|
||||
|
||||
await target.dumpAllCollectionsToDir('./backups/safe-backup');
|
||||
console.log('✅ Backup successful');
|
||||
|
||||
} catch (error) {
|
||||
console.error('❌ Backup failed:', error);
|
||||
// Implement notification/retry logic
|
||||
|
||||
} finally {
|
||||
await mongoDump.stop();
|
||||
}
|
||||
```
|
||||
|
||||
### Clean Shutdown
|
||||
|
||||
Always ensure proper cleanup of database connections:
|
||||
|
||||
```typescript
|
||||
// Closes all open MongoDB connections
|
||||
await mongoDump.stop();
|
||||
```
|
||||
|
||||
## API Reference 🔧
|
||||
|
||||
### MongoDump
|
||||
|
||||
The main class for managing dump operations.
|
||||
|
||||
**Methods:**
|
||||
- `addMongoTargetByMongoDescriptor(descriptor: IMongoDescriptor)`: Create a new dump target
|
||||
- `stop()`: Close all database connections
|
||||
|
||||
### MongoDumpTarget
|
||||
|
||||
Handles operations for a specific MongoDB database.
|
||||
|
||||
**Methods:**
|
||||
- `getCollections()`: List all collections in the database
|
||||
- `dumpCollectionToDir(collection, directory, nameTransform?)`: Dump single collection
|
||||
- `dumpAllCollectionsToDir(directory, nameTransform?, cleanDir?)`: Dump all collections
|
||||
- `dumpCollectionToTarArchiveFile(collection, filepath)`: Create tar archive
|
||||
- `dumpAllCollectionsToTarArchiveStream()`: Get tar archive stream
|
||||
|
||||
## Best Practices 💡
|
||||
|
||||
1. **Environment Variables**: Store credentials in environment variables, never hardcode them
|
||||
2. **Error Handling**: Always wrap dump operations in try-catch blocks
|
||||
3. **Connection Cleanup**: Always call `stop()` when done
|
||||
4. **Backup Verification**: Periodically verify your backups can be restored
|
||||
5. **Storage Rotation**: Implement backup rotation to manage disk space
|
||||
6. **Monitoring**: Add logging and monitoring for production backup jobs
|
||||
|
||||
## Why Choose @push.rocks/mongodump? 🌟
|
||||
|
||||
- **Type Safety**: Full TypeScript support with comprehensive type definitions
|
||||
- **Modern Async**: Built on promises and async/await patterns
|
||||
- **Flexible Export**: Multiple export formats and naming options
|
||||
- **Production Ready**: Battle-tested in production environments
|
||||
- **Clean API**: Intuitive, well-documented interface
|
||||
- **Active Maintenance**: Regular updates and security patches
|
||||
## License and Legal Information
|
||||
|
||||
This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository.
|
||||
|
||||
Reference in New Issue
Block a user