4 Commits
v5.1.1 ... main

Author SHA1 Message Date
d437ffc226 v5.3.0
Some checks failed
Default (tags) / security (push) Successful in 37s
Default (tags) / test (push) Failing after 26s
Default (tags) / release (push) Has been skipped
Default (tags) / metadata (push) Has been skipped
2026-02-17 16:50:04 +00:00
e36758f183 feat(auth): add AWS SigV4 authentication and bucket policy support 2026-02-17 16:50:04 +00:00
adf45dce2d v5.2.0
Some checks failed
Default (tags) / security (push) Successful in 40s
Default (tags) / test (push) Failing after 27s
Default (tags) / release (push) Has been skipped
Default (tags) / metadata (push) Has been skipped
2026-02-17 16:28:50 +00:00
eb232b6e8e feat(auth,policy): add AWS SigV4 authentication and S3 bucket policy support 2026-02-17 16:28:50 +00:00
21 changed files with 2684 additions and 136 deletions

View File

@@ -1,5 +1,24 @@
# Changelog # Changelog
## 2026-02-17 - 5.3.0 - feat(auth)
add AWS SigV4 authentication and bucket policy support
- Implement AWS SigV4 full verification (constant-time comparison, 15-minute clock skew enforcement) and expose default signing region (server.region = 'us-east-1').
- Add IAM-style bucket policy engine with Put/Get/Delete policy APIs (GetBucketPolicy/PutBucketPolicy/DeleteBucketPolicy), wildcard action/resource matching, Allow/Deny evaluation, and on-disk persistence under .policies/{bucket}.policy.json.
- Documentation and README expanded with policy usage, examples, API table entries, and notes about policy CRUD and behavior for anonymous/authenticated requests.
- Rust code refactors: simplify storage/server result structs and multipart handling (removed several unused size/key/bucket fields), remove S3Error::to_response and error_xml helpers, and other internal cleanup to support new auth/policy features.
## 2026-02-17 - 5.2.0 - feat(auth,policy)
add AWS SigV4 authentication and S3 bucket policy support
- Implemented real AWS SigV4 verification (HMAC-SHA256), including x-amz-date handling, clock skew enforcement and constant-time signature comparison
- Added bucket policy model, validator and evaluation engine (Deny > Allow > NoOpinion) with a PolicyStore (RwLock cache + disk-backed .policies/*.policy.json)
- Integrated action resolution and auth+policy pipeline into the HTTP server: authorization checks run per-request, anonymous requests are denied by default, ListAllMyBuckets requires authentication
- Added bucket policy CRUD handlers via ?policy query parameter (GET/PUT/DELETE) and cleanup of policies on bucket deletion
- Storage and config updates: created .policies dir and policy path helpers; default region added to server config (TS + Rust)
- Added comprehensive tests for auth and policy behavior (policy CRUD, evaluation, per-action enforcement, auth integration)
- Updated Rust dependencies and Cargo.toml/Cargo.lock to include hmac, sha2, hex, subtle, cpufeatures
## 2026-02-13 - 5.1.1 - fix(smarts3) ## 2026-02-13 - 5.1.1 - fix(smarts3)
replace TypeScript server with Rust-powered core and IPC bridge replace TypeScript server with Rust-powered core and IPC bridge

View File

@@ -1,6 +1,6 @@
{ {
"name": "@push.rocks/smarts3", "name": "@push.rocks/smarts3",
"version": "5.1.1", "version": "5.3.0",
"private": false, "private": false,
"description": "A Node.js TypeScript package to create a local S3 endpoint for simulating AWS S3 operations using mapped local directories for development and testing purposes.", "description": "A Node.js TypeScript package to create a local S3 endpoint for simulating AWS S3 operations using mapped local directories for development and testing purposes.",
"main": "dist_ts/index.js", "main": "dist_ts/index.js",

View File

@@ -7,7 +7,8 @@
- TypeScript is thin IPC wrapper; all HTTP/storage/routing in Rust binary `rusts3` - TypeScript is thin IPC wrapper; all HTTP/storage/routing in Rust binary `rusts3`
- Full S3 compatibility: PUT, GET, HEAD, DELETE for objects and buckets - Full S3 compatibility: PUT, GET, HEAD, DELETE for objects and buckets
- Multipart upload support (streaming, no OOM) - Multipart upload support (streaming, no OOM)
- Authentication (AWS v2/v4 signature key extraction) - **Real AWS SigV4 authentication** (cryptographic signature verification)
- **Bucket policies** (AWS/MinIO-compatible JSON policies, public access support)
- CORS support - CORS support
- ListBuckets, ListObjects (v1/v2), CopyObject - ListBuckets, ListObjects (v1/v2), CopyObject
@@ -15,12 +16,15 @@
### Rust Binary (`rust/src/`) ### Rust Binary (`rust/src/`)
- `main.rs` - Clap CLI, management mode entry - `main.rs` - Clap CLI, management mode entry
- `config.rs` - Serde config structs matching TS interfaces - `config.rs` - Serde config structs matching TS interfaces (includes `region`)
- `management.rs` - IPC loop (newline-delimited JSON over stdin/stdout) - `management.rs` - IPC loop (newline-delimited JSON over stdin/stdout)
- `server.rs` - hyper 1.x HTTP server, routing, CORS, auth, all S3 handlers - `server.rs` - hyper 1.x HTTP server, routing, CORS, auth+policy pipeline, all S3 handlers
- `storage.rs` - FileStore: filesystem-backed storage, multipart manager - `storage.rs` - FileStore: filesystem-backed storage, multipart manager, `.policies/` dir
- `xml_response.rs` - S3 XML response builders - `xml_response.rs` - S3 XML response builders
- `s3_error.rs` - S3 error codes with HTTP status mapping - `s3_error.rs` - S3 error codes with HTTP status mapping
- `auth.rs` - AWS SigV4 signature verification (HMAC-SHA256, clock skew, constant-time compare)
- `action.rs` - S3Action enum + request-to-IAM-action resolver + RequestContext
- `policy.rs` - BucketPolicy model, evaluation engine (Deny > Allow > NoOpinion), PolicyStore (RwLock cache + disk)
### TypeScript Bridge (`ts/`) ### TypeScript Bridge (`ts/`)
- `ts/index.ts` - Smarts3 class with RustBridge<TRustS3Commands> - `ts/index.ts` - Smarts3 class with RustBridge<TRustS3Commands>
@@ -39,6 +43,7 @@
- Metadata: `{root}/{bucket}/{key}._S3_object.metadata.json` - Metadata: `{root}/{bucket}/{key}._S3_object.metadata.json`
- MD5: `{root}/{bucket}/{key}._S3_object.md5` - MD5: `{root}/{bucket}/{key}._S3_object.md5`
- Multipart: `{root}/.multipart/{upload_id}/part-{N}` - Multipart: `{root}/.multipart/{upload_id}/part-{N}`
- Policies: `{root}/.policies/{bucket}.policy.json`
## Build ## Build
@@ -55,6 +60,10 @@
## Testing ## Testing
- `test/test.aws-sdk.node.ts` - AWS SDK v3 compatibility (10 tests) - `test/test.aws-sdk.node.ts` - AWS SDK v3 compatibility (10 tests, auth disabled, port 3337)
- `test/test.auth.node.ts` - Auth + bucket policy integration (20 tests, auth enabled, port 3344)
- `test/test.policy-crud.node.ts` - Policy API CRUD + validation edge cases (17 tests, port 3345)
- `test/test.policy-eval.node.ts` - Policy evaluation: principals, actions, resources, deny-vs-allow (22 tests, port 3346)
- `test/test.policy-actions.node.ts` - Per-action policy enforcement (15 tests, port 3347)
- `test/test.ts` - SmartBucket integration (3 tests) - `test/test.ts` - SmartBucket integration (3 tests)
- Run: `pnpm test` or `tstest test/test.aws-sdk.node.ts --verbose` - Run: `pnpm test` or `tstest test/test.aws-sdk.node.ts --verbose`

View File

@@ -16,7 +16,8 @@ For reporting bugs, issues, or security vulnerabilities, please visit [community
| Range requests | ✅ Seek-based | ✅ | ❌ Full read | | Range requests | ✅ Seek-based | ✅ | ❌ Full read |
| Language | Rust + TypeScript | Go | JavaScript | | Language | Rust + TypeScript | Go | JavaScript |
| Multipart uploads | ✅ Full support | ✅ | ❌ | | Multipart uploads | ✅ Full support | ✅ | ❌ |
| Auth | AWS v2/v4 key extraction | Full IAM | Basic | | Auth | AWS SigV4 (full verification) | Full IAM | Basic |
| Bucket policies | ✅ IAM-style evaluation | ✅ | ❌ |
### Core Features ### Core Features
@@ -25,7 +26,8 @@ For reporting bugs, issues, or security vulnerabilities, please visit [community
- 📂 **Filesystem-backed storage** — buckets map to directories, objects to files - 📂 **Filesystem-backed storage** — buckets map to directories, objects to files
- 📤 **Streaming multipart uploads** — large files without memory pressure - 📤 **Streaming multipart uploads** — large files without memory pressure
- 🎯 **Byte-range requests**`seek()` directly to the requested byte offset - 🎯 **Byte-range requests**`seek()` directly to the requested byte offset
- 🔐 **Authentication**AWS v2/v4 signature key extraction - 🔐 **AWS SigV4 authentication**full signature verification with constant-time comparison and 15-min clock skew enforcement
- 📜 **Bucket policies** — IAM-style JSON policies with Allow/Deny evaluation, wildcard matching, and anonymous access support
- 🌐 **CORS middleware** — configurable cross-origin support - 🌐 **CORS middleware** — configurable cross-origin support
- 📊 **Structured logging** — tracing-based, error through debug levels - 📊 **Structured logging** — tracing-based, error through debug levels
- 🧹 **Clean slate mode** — wipe storage on startup for test isolation - 🧹 **Clean slate mode** — wipe storage on startup for test isolation
@@ -73,6 +75,7 @@ const config: ISmarts3Config = {
port: 3000, // Default: 3000 port: 3000, // Default: 3000
address: '0.0.0.0', // Default: '0.0.0.0' address: '0.0.0.0', // Default: '0.0.0.0'
silent: false, // Default: false silent: false, // Default: false
region: 'us-east-1', // Default: 'us-east-1' — used for SigV4 signing
}, },
storage: { storage: {
directory: './my-data', // Default: .nogit/bucketsDir directory: './my-data', // Default: .nogit/bucketsDir
@@ -241,6 +244,56 @@ await client.send(new CompleteMultipartUploadCommand({
})); }));
``` ```
## 📜 Bucket Policies
smarts3 supports AWS-style bucket policies for fine-grained access control. Policies use the same IAM JSON format as real S3 — so you can develop and test your policy logic locally before deploying.
When `auth.enabled` is `true`, the auth pipeline works as follows:
1. **Authenticate** — verify the AWS SigV4 signature (anonymous requests skip this step)
2. **Authorize** — evaluate bucket policies against the request action, resource, and caller identity
3. **Default** — authenticated users get full access; anonymous requests are denied unless a policy explicitly allows them
### Setting a Bucket Policy
Use the S3 `PutBucketPolicy` API (or any S3 client that supports it):
```typescript
import { PutBucketPolicyCommand } from '@aws-sdk/client-s3';
// Allow anonymous read access to all objects in a bucket
await client.send(new PutBucketPolicyCommand({
Bucket: 'public-assets',
Policy: JSON.stringify({
Version: '2012-10-17',
Statement: [{
Sid: 'PublicRead',
Effect: 'Allow',
Principal: '*',
Action: ['s3:GetObject'],
Resource: ['arn:aws:s3:::public-assets/*'],
}],
}),
}));
```
### Policy Features
- **Effect**: `Allow` and `Deny` (explicit Deny always wins)
- **Principal**: `"*"` (everyone) or `{ "AWS": ["arn:..."] }` for specific identities
- **Action**: IAM-style actions like `s3:GetObject`, `s3:PutObject`, `s3:*`, or prefix wildcards like `s3:Get*`
- **Resource**: ARN patterns with `*` and `?` wildcards (e.g. `arn:aws:s3:::my-bucket/*`)
- **Persistence**: Policies survive server restarts — stored as JSON on disk alongside your data
### Policy CRUD Operations
| Operation | AWS SDK Command | HTTP |
|-----------|----------------|------|
| Get policy | `GetBucketPolicyCommand` | `GET /{bucket}?policy` |
| Set policy | `PutBucketPolicyCommand` | `PUT /{bucket}?policy` |
| Delete policy | `DeleteBucketPolicyCommand` | `DELETE /{bucket}?policy` |
Deleting a bucket automatically removes its associated policy.
## 🧪 Testing Integration ## 🧪 Testing Integration
```typescript ```typescript
@@ -314,7 +367,8 @@ smarts3 uses a **hybrid Rust + TypeScript** architecture:
│ ├─ S3 path-style routing │ │ ├─ S3 path-style routing │
│ ├─ Streaming storage layer │ │ ├─ Streaming storage layer │
│ ├─ Multipart manager │ │ ├─ Multipart manager │
│ ├─ CORS / Auth middleware │ ├─ SigV4 auth + policy engine
│ ├─ CORS middleware │
│ └─ S3 XML response builder │ │ └─ S3 XML response builder │
├─────────────────────────────────┤ ├─────────────────────────────────┤
│ TypeScript (thin IPC wrapper) │ │ TypeScript (thin IPC wrapper) │
@@ -347,6 +401,9 @@ smarts3 uses a **hybrid Rust + TypeScript** architecture:
| CompleteMultipartUpload | `POST /{bucket}/{key}?uploadId` | | | CompleteMultipartUpload | `POST /{bucket}/{key}?uploadId` | |
| AbortMultipartUpload | `DELETE /{bucket}/{key}?uploadId` | | | AbortMultipartUpload | `DELETE /{bucket}/{key}?uploadId` | |
| ListMultipartUploads | `GET /{bucket}?uploads` | | | ListMultipartUploads | `GET /{bucket}?uploads` | |
| GetBucketPolicy | `GET /{bucket}?policy` | |
| PutBucketPolicy | `PUT /{bucket}?policy` | |
| DeleteBucketPolicy | `DELETE /{bucket}?policy` | |
### On-Disk Format ### On-Disk Format
@@ -362,6 +419,8 @@ smarts3 uses a **hybrid Rust + TypeScript** architecture:
part-1 # Part data files part-1 # Part data files
part-2 part-2
... ...
.policies/
{bucket}.policy.json # Bucket policy (IAM JSON format)
``` ```
## 🔗 Related Packages ## 🔗 Related Packages

45
rust/Cargo.lock generated
View File

@@ -197,6 +197,15 @@ version = "0.8.7"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b" checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b"
[[package]]
name = "cpufeatures"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280"
dependencies = [
"libc",
]
[[package]] [[package]]
name = "crypto-common" name = "crypto-common"
version = "0.1.7" version = "0.1.7"
@@ -215,6 +224,7 @@ checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292"
dependencies = [ dependencies = [
"block-buffer", "block-buffer",
"crypto-common", "crypto-common",
"subtle",
] ]
[[package]] [[package]]
@@ -302,6 +312,21 @@ version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea" checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "hex"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70"
[[package]]
name = "hmac"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c49c37c09c17a53d937dfbb742eb3a961d65a994e6bcdcf37e7399d0cc8ab5e"
dependencies = [
"digest",
]
[[package]] [[package]]
name = "http" name = "http"
version = "1.4.0" version = "1.4.0"
@@ -748,6 +773,8 @@ dependencies = [
"chrono", "chrono",
"clap", "clap",
"futures-core", "futures-core",
"hex",
"hmac",
"http-body-util", "http-body-util",
"hyper", "hyper",
"hyper-util", "hyper-util",
@@ -756,6 +783,7 @@ dependencies = [
"quick-xml", "quick-xml",
"serde", "serde",
"serde_json", "serde_json",
"sha2",
"thiserror", "thiserror",
"tokio", "tokio",
"tokio-util", "tokio-util",
@@ -820,6 +848,17 @@ dependencies = [
"zmij", "zmij",
] ]
[[package]]
name = "sha2"
version = "0.10.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283"
dependencies = [
"cfg-if",
"cpufeatures",
"digest",
]
[[package]] [[package]]
name = "sharded-slab" name = "sharded-slab"
version = "0.1.7" version = "0.1.7"
@@ -873,6 +912,12 @@ version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f" checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f"
[[package]]
name = "subtle"
version = "2.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292"
[[package]] [[package]]
name = "syn" name = "syn"
version = "2.0.115" version = "2.0.115"

View File

@@ -28,3 +28,6 @@ percent-encoding = "2"
url = "2" url = "2"
chrono = { version = "0.4", features = ["serde"] } chrono = { version = "0.4", features = ["serde"] }
futures-core = "0.3" futures-core = "0.3"
hmac = "0.12"
sha2 = "0.10"
hex = "0.4"

172
rust/src/action.rs Normal file
View File

@@ -0,0 +1,172 @@
use hyper::body::Incoming;
use hyper::{Method, Request};
use std::collections::HashMap;
/// S3 actions that map to IAM permission strings.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum S3Action {
ListAllMyBuckets,
CreateBucket,
DeleteBucket,
HeadBucket,
ListBucket,
GetObject,
HeadObject,
PutObject,
DeleteObject,
CopyObject,
ListBucketMultipartUploads,
AbortMultipartUpload,
InitiateMultipartUpload,
UploadPart,
CompleteMultipartUpload,
GetBucketPolicy,
PutBucketPolicy,
DeleteBucketPolicy,
}
impl S3Action {
/// Return the IAM-style action string (e.g. "s3:GetObject").
pub fn iam_action(&self) -> &'static str {
match self {
S3Action::ListAllMyBuckets => "s3:ListAllMyBuckets",
S3Action::CreateBucket => "s3:CreateBucket",
S3Action::DeleteBucket => "s3:DeleteBucket",
S3Action::HeadBucket => "s3:ListBucket",
S3Action::ListBucket => "s3:ListBucket",
S3Action::GetObject => "s3:GetObject",
S3Action::HeadObject => "s3:GetObject",
S3Action::PutObject => "s3:PutObject",
S3Action::DeleteObject => "s3:DeleteObject",
S3Action::CopyObject => "s3:PutObject",
S3Action::ListBucketMultipartUploads => "s3:ListBucketMultipartUploads",
S3Action::AbortMultipartUpload => "s3:AbortMultipartUpload",
S3Action::InitiateMultipartUpload => "s3:PutObject",
S3Action::UploadPart => "s3:PutObject",
S3Action::CompleteMultipartUpload => "s3:PutObject",
S3Action::GetBucketPolicy => "s3:GetBucketPolicy",
S3Action::PutBucketPolicy => "s3:PutBucketPolicy",
S3Action::DeleteBucketPolicy => "s3:DeleteBucketPolicy",
}
}
}
/// Context extracted from a request, used for policy evaluation.
#[derive(Debug, Clone)]
pub struct RequestContext {
pub action: S3Action,
pub bucket: Option<String>,
pub key: Option<String>,
}
impl RequestContext {
/// Build the ARN for this request's resource.
pub fn resource_arn(&self) -> String {
match (&self.bucket, &self.key) {
(Some(bucket), Some(key)) => format!("arn:aws:s3:::{}/{}", bucket, key),
(Some(bucket), None) => format!("arn:aws:s3:::{}", bucket),
_ => "arn:aws:s3:::*".to_string(),
}
}
}
/// Resolve the S3 action from an incoming HTTP request.
pub fn resolve_action(req: &Request<Incoming>) -> RequestContext {
let method = req.method().clone();
let path = req.uri().path().to_string();
let query_string = req.uri().query().unwrap_or("").to_string();
let query = parse_query_simple(&query_string);
let segments: Vec<&str> = path
.trim_start_matches('/')
.splitn(2, '/')
.filter(|s| !s.is_empty())
.collect();
match segments.len() {
0 => {
// Root: GET / -> ListBuckets
RequestContext {
action: S3Action::ListAllMyBuckets,
bucket: None,
key: None,
}
}
1 => {
let bucket = percent_decode(segments[0]);
let has_policy = query.contains_key("policy");
let has_uploads = query.contains_key("uploads");
let action = match (&method, has_policy, has_uploads) {
(&Method::GET, true, _) => S3Action::GetBucketPolicy,
(&Method::PUT, true, _) => S3Action::PutBucketPolicy,
(&Method::DELETE, true, _) => S3Action::DeleteBucketPolicy,
(&Method::GET, _, true) => S3Action::ListBucketMultipartUploads,
(&Method::GET, _, _) => S3Action::ListBucket,
(&Method::PUT, _, _) => S3Action::CreateBucket,
(&Method::DELETE, _, _) => S3Action::DeleteBucket,
(&Method::HEAD, _, _) => S3Action::HeadBucket,
_ => S3Action::ListBucket,
};
RequestContext {
action,
bucket: Some(bucket),
key: None,
}
}
2 => {
let bucket = percent_decode(segments[0]);
let key = percent_decode(segments[1]);
let has_copy_source = req.headers().contains_key("x-amz-copy-source");
let has_part_number = query.contains_key("partNumber");
let has_upload_id = query.contains_key("uploadId");
let has_uploads = query.contains_key("uploads");
let action = match &method {
&Method::PUT if has_part_number && has_upload_id => S3Action::UploadPart,
&Method::PUT if has_copy_source => S3Action::CopyObject,
&Method::PUT => S3Action::PutObject,
&Method::GET => S3Action::GetObject,
&Method::HEAD => S3Action::HeadObject,
&Method::DELETE if has_upload_id => S3Action::AbortMultipartUpload,
&Method::DELETE => S3Action::DeleteObject,
&Method::POST if has_uploads => S3Action::InitiateMultipartUpload,
&Method::POST if has_upload_id => S3Action::CompleteMultipartUpload,
_ => S3Action::GetObject,
};
RequestContext {
action,
bucket: Some(bucket),
key: Some(key),
}
}
_ => RequestContext {
action: S3Action::ListAllMyBuckets,
bucket: None,
key: None,
},
}
}
fn parse_query_simple(query_string: &str) -> HashMap<String, String> {
let mut map = HashMap::new();
if query_string.is_empty() {
return map;
}
for pair in query_string.split('&') {
let mut parts = pair.splitn(2, '=');
let key = parts.next().unwrap_or("");
let value = parts.next().unwrap_or("");
map.insert(key.to_string(), value.to_string());
}
map
}
fn percent_decode(s: &str) -> String {
percent_encoding::percent_decode_str(s)
.decode_utf8_lossy()
.to_string()
}

310
rust/src/auth.rs Normal file
View File

@@ -0,0 +1,310 @@
use hmac::{Hmac, Mac};
use hyper::body::Incoming;
use hyper::Request;
use sha2::{Digest, Sha256};
use std::collections::HashMap;
use crate::config::{Credential, S3Config};
use crate::s3_error::S3Error;
type HmacSha256 = Hmac<Sha256>;
/// The identity of an authenticated caller.
#[derive(Debug, Clone)]
pub struct AuthenticatedIdentity {
pub access_key_id: String,
}
/// Parsed components of an AWS4-HMAC-SHA256 Authorization header.
struct SigV4Header {
access_key_id: String,
date_stamp: String,
region: String,
signed_headers: Vec<String>,
signature: String,
}
/// Verify the request's SigV4 signature. Returns the caller identity on success.
pub fn verify_request(
req: &Request<Incoming>,
config: &S3Config,
) -> Result<AuthenticatedIdentity, S3Error> {
let auth_header = req
.headers()
.get("authorization")
.and_then(|v| v.to_str().ok())
.unwrap_or("");
// Reject SigV2
if auth_header.starts_with("AWS ") {
return Err(S3Error::authorization_header_malformed());
}
if !auth_header.starts_with("AWS4-HMAC-SHA256") {
return Err(S3Error::authorization_header_malformed());
}
let parsed = parse_auth_header(auth_header)?;
// Look up credential
let credential = find_credential(&parsed.access_key_id, config)
.ok_or_else(S3Error::invalid_access_key_id)?;
// Get x-amz-date
let amz_date = req
.headers()
.get("x-amz-date")
.and_then(|v| v.to_str().ok())
.or_else(|| {
req.headers()
.get("date")
.and_then(|v| v.to_str().ok())
})
.ok_or_else(|| S3Error::missing_security_header("Missing x-amz-date header"))?;
// Enforce 15-min clock skew
check_clock_skew(amz_date)?;
// Get payload hash
let content_sha256 = req
.headers()
.get("x-amz-content-sha256")
.and_then(|v| v.to_str().ok())
.unwrap_or("UNSIGNED-PAYLOAD");
// Build canonical request
let canonical_request = build_canonical_request(req, &parsed.signed_headers, content_sha256);
// Build string to sign
let scope = format!(
"{}/{}/s3/aws4_request",
parsed.date_stamp, parsed.region
);
let canonical_hash = hex::encode(Sha256::digest(canonical_request.as_bytes()));
let string_to_sign = format!(
"AWS4-HMAC-SHA256\n{}\n{}\n{}",
amz_date, scope, canonical_hash
);
// Derive signing key
let signing_key = derive_signing_key(
&credential.secret_access_key,
&parsed.date_stamp,
&parsed.region,
);
// Compute signature
let computed = hmac_sha256(&signing_key, string_to_sign.as_bytes());
let computed_hex = hex::encode(&computed);
// Constant-time comparison
if !constant_time_eq(computed_hex.as_bytes(), parsed.signature.as_bytes()) {
return Err(S3Error::signature_does_not_match());
}
Ok(AuthenticatedIdentity {
access_key_id: parsed.access_key_id,
})
}
/// Parse the Authorization header into its components.
fn parse_auth_header(header: &str) -> Result<SigV4Header, S3Error> {
// Format: AWS4-HMAC-SHA256 Credential=KEY/YYYYMMDD/region/s3/aws4_request, SignedHeaders=h1;h2, Signature=hex
let after_algo = header
.strip_prefix("AWS4-HMAC-SHA256")
.ok_or_else(S3Error::authorization_header_malformed)?
.trim();
let mut credential_str = None;
let mut signed_headers_str = None;
let mut signature_str = None;
for part in after_algo.split(',') {
let part = part.trim();
if let Some(val) = part.strip_prefix("Credential=") {
credential_str = Some(val.trim());
} else if let Some(val) = part.strip_prefix("SignedHeaders=") {
signed_headers_str = Some(val.trim());
} else if let Some(val) = part.strip_prefix("Signature=") {
signature_str = Some(val.trim());
}
}
let credential_str = credential_str
.ok_or_else(S3Error::authorization_header_malformed)?;
let signed_headers_str = signed_headers_str
.ok_or_else(S3Error::authorization_header_malformed)?;
let signature = signature_str
.ok_or_else(S3Error::authorization_header_malformed)?
.to_string();
// Parse credential: KEY/YYYYMMDD/region/s3/aws4_request
let cred_parts: Vec<&str> = credential_str.splitn(5, '/').collect();
if cred_parts.len() < 5 {
return Err(S3Error::authorization_header_malformed());
}
let access_key_id = cred_parts[0].to_string();
let date_stamp = cred_parts[1].to_string();
let region = cred_parts[2].to_string();
let signed_headers: Vec<String> = signed_headers_str
.split(';')
.map(|s| s.trim().to_lowercase())
.collect();
Ok(SigV4Header {
access_key_id,
date_stamp,
region,
signed_headers,
signature,
})
}
/// Find a credential by access key ID.
fn find_credential<'a>(access_key_id: &str, config: &'a S3Config) -> Option<&'a Credential> {
config
.auth
.credentials
.iter()
.find(|c| c.access_key_id == access_key_id)
}
/// Check clock skew (15 minutes max).
fn check_clock_skew(amz_date: &str) -> Result<(), S3Error> {
// Parse ISO 8601 basic format: YYYYMMDDTHHMMSSZ
let parsed = chrono::NaiveDateTime::parse_from_str(amz_date, "%Y%m%dT%H%M%SZ")
.map_err(|_| S3Error::authorization_header_malformed())?;
let request_time = chrono::DateTime::<chrono::Utc>::from_naive_utc_and_offset(parsed, chrono::Utc);
let now = chrono::Utc::now();
let diff = (now - request_time).num_seconds().unsigned_abs();
if diff > 15 * 60 {
return Err(S3Error::request_time_too_skewed());
}
Ok(())
}
/// Build the canonical request string.
fn build_canonical_request(
req: &Request<Incoming>,
signed_headers: &[String],
payload_hash: &str,
) -> String {
let method = req.method().as_str();
let uri_path = req.uri().path();
// Canonical URI: the path, already percent-encoded by the client
let canonical_uri = if uri_path.is_empty() { "/" } else { uri_path };
// Canonical query string: sorted key=value pairs
let canonical_query = build_canonical_query(req.uri().query().unwrap_or(""));
// Canonical headers: sorted by lowercase header name
let canonical_headers = build_canonical_headers(req, signed_headers);
// Signed headers string
let signed_headers_str = signed_headers.join(";");
// Payload hash — accept UNSIGNED-PAYLOAD and STREAMING-AWS4-HMAC-SHA256-PAYLOAD as-is
let effective_payload_hash = if payload_hash == "UNSIGNED-PAYLOAD"
|| payload_hash == "STREAMING-AWS4-HMAC-SHA256-PAYLOAD"
{
payload_hash.to_string()
} else {
payload_hash.to_string()
};
format!(
"{}\n{}\n{}\n{}\n{}\n{}",
method,
canonical_uri,
canonical_query,
canonical_headers,
signed_headers_str,
effective_payload_hash
)
}
/// Build canonical query string (sorted key=value pairs).
fn build_canonical_query(query: &str) -> String {
if query.is_empty() {
return String::new();
}
let mut pairs: Vec<(String, String)> = Vec::new();
for pair in query.split('&') {
let mut parts = pair.splitn(2, '=');
let key = parts.next().unwrap_or("");
let value = parts.next().unwrap_or("");
pairs.push((key.to_string(), value.to_string()));
}
pairs.sort();
pairs
.iter()
.map(|(k, v)| format!("{}={}", k, v))
.collect::<Vec<_>>()
.join("&")
}
/// Build canonical headers string.
fn build_canonical_headers(req: &Request<Incoming>, signed_headers: &[String]) -> String {
let mut header_map: HashMap<String, Vec<String>> = HashMap::new();
for (name, value) in req.headers() {
let name_lower = name.as_str().to_lowercase();
if signed_headers.contains(&name_lower) {
if let Ok(val) = value.to_str() {
header_map
.entry(name_lower)
.or_default()
.push(val.trim().to_string());
}
}
}
let mut result = String::new();
for header_name in signed_headers {
let values = header_map
.get(header_name)
.map(|v| v.join(","))
.unwrap_or_default();
result.push_str(header_name);
result.push(':');
result.push_str(&values);
result.push('\n');
}
result
}
/// Derive the signing key via 4-step HMAC chain.
fn derive_signing_key(secret_key: &str, date_stamp: &str, region: &str) -> Vec<u8> {
let k_secret = format!("AWS4{}", secret_key);
let k_date = hmac_sha256(k_secret.as_bytes(), date_stamp.as_bytes());
let k_region = hmac_sha256(&k_date, region.as_bytes());
let k_service = hmac_sha256(&k_region, b"s3");
hmac_sha256(&k_service, b"aws4_request")
}
/// Compute HMAC-SHA256.
fn hmac_sha256(key: &[u8], data: &[u8]) -> Vec<u8> {
let mut mac = HmacSha256::new_from_slice(key).expect("HMAC key length is always valid");
mac.update(data);
mac.finalize().into_bytes().to_vec()
}
/// Constant-time byte comparison.
fn constant_time_eq(a: &[u8], b: &[u8]) -> bool {
if a.len() != b.len() {
return false;
}
let mut diff = 0u8;
for (x, y) in a.iter().zip(b.iter()) {
diff |= x ^ y;
}
diff == 0
}

View File

@@ -18,6 +18,12 @@ pub struct ServerConfig {
pub port: u16, pub port: u16,
pub address: String, pub address: String,
pub silent: bool, pub silent: bool,
#[serde(default = "default_region")]
pub region: String,
}
fn default_region() -> String {
"us-east-1".to_string()
} }
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]

View File

@@ -1,5 +1,8 @@
mod action;
mod auth;
mod config; mod config;
mod management; mod management;
mod policy;
mod s3_error; mod s3_error;
mod server; mod server;
mod storage; mod storage;

429
rust/src/policy.rs Normal file
View File

@@ -0,0 +1,429 @@
use serde::{Deserialize, Deserializer, Serialize};
use std::collections::HashMap;
use std::path::PathBuf;
use tokio::fs;
use tokio::sync::RwLock;
use crate::action::RequestContext;
use crate::auth::AuthenticatedIdentity;
use crate::s3_error::S3Error;
// ============================
// Policy data model
// ============================
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct BucketPolicy {
#[serde(rename = "Version")]
pub version: String,
#[serde(rename = "Statement")]
pub statements: Vec<PolicyStatement>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct PolicyStatement {
#[serde(rename = "Sid", default, skip_serializing_if = "Option::is_none")]
pub sid: Option<String>,
#[serde(rename = "Effect")]
pub effect: PolicyEffect,
#[serde(rename = "Principal", deserialize_with = "deserialize_principal")]
pub principal: Principal,
#[serde(rename = "Action", deserialize_with = "deserialize_string_or_vec")]
pub action: Vec<String>,
#[serde(rename = "Resource", deserialize_with = "deserialize_string_or_vec")]
pub resource: Vec<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
pub enum PolicyEffect {
Allow,
Deny,
}
#[derive(Debug, Clone)]
pub enum Principal {
Wildcard,
Aws(Vec<String>),
}
impl Serialize for Principal {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: serde::Serializer,
{
match self {
Principal::Wildcard => serializer.serialize_str("*"),
Principal::Aws(ids) => {
use serde::ser::SerializeMap;
let mut map = serializer.serialize_map(Some(1))?;
if ids.len() == 1 {
map.serialize_entry("AWS", &ids[0])?;
} else {
map.serialize_entry("AWS", ids)?;
}
map.end()
}
}
}
}
fn deserialize_principal<'de, D>(deserializer: D) -> Result<Principal, D::Error>
where
D: Deserializer<'de>,
{
#[derive(Deserialize)]
#[serde(untagged)]
enum PrincipalRaw {
Star(String),
Map(HashMap<String, StringOrVec>),
}
let raw = PrincipalRaw::deserialize(deserializer)?;
match raw {
PrincipalRaw::Star(s) if s == "*" => Ok(Principal::Wildcard),
PrincipalRaw::Star(_) => Err(serde::de::Error::custom(
"Principal string must be \"*\"",
)),
PrincipalRaw::Map(map) => {
if let Some(aws) = map.get("AWS") {
Ok(Principal::Aws(aws.clone().into_vec()))
} else {
Err(serde::de::Error::custom("Principal map must contain \"AWS\" key"))
}
}
}
}
#[derive(Debug, Clone, Deserialize)]
#[serde(untagged)]
enum StringOrVec {
Single(String),
Multiple(Vec<String>),
}
impl StringOrVec {
fn into_vec(self) -> Vec<String> {
match self {
StringOrVec::Single(s) => vec![s],
StringOrVec::Multiple(v) => v,
}
}
}
fn deserialize_string_or_vec<'de, D>(deserializer: D) -> Result<Vec<String>, D::Error>
where
D: Deserializer<'de>,
{
let raw = StringOrVec::deserialize(deserializer)?;
Ok(raw.into_vec())
}
// ============================
// Policy evaluation
// ============================
#[derive(Debug, Clone, PartialEq)]
pub enum PolicyDecision {
Allow,
Deny,
NoOpinion,
}
/// Evaluate a bucket policy against a request context and caller identity.
pub fn evaluate_policy(
policy: &BucketPolicy,
ctx: &RequestContext,
identity: Option<&AuthenticatedIdentity>,
) -> PolicyDecision {
let resource_arn = ctx.resource_arn();
let iam_action = ctx.action.iam_action();
let mut has_allow = false;
for stmt in &policy.statements {
// Check principal match
if !principal_matches(&stmt.principal, identity) {
continue;
}
// Check action match
if !action_matches(&stmt.action, iam_action) {
continue;
}
// Check resource match
if !resource_matches(&stmt.resource, &resource_arn, ctx.bucket.as_deref()) {
continue;
}
// Statement matches — apply effect
match stmt.effect {
PolicyEffect::Deny => return PolicyDecision::Deny,
PolicyEffect::Allow => has_allow = true,
}
}
if has_allow {
PolicyDecision::Allow
} else {
PolicyDecision::NoOpinion
}
}
/// Check if the principal matches the caller.
fn principal_matches(principal: &Principal, identity: Option<&AuthenticatedIdentity>) -> bool {
match principal {
Principal::Wildcard => true,
Principal::Aws(ids) => {
if let Some(id) = identity {
ids.iter().any(|arn| {
// Match against full ARN or just the access key ID
arn == "*" || arn.ends_with(&id.access_key_id)
})
} else {
false
}
}
}
}
/// Check if the action matches. Supports wildcard `s3:*` and `*`.
fn action_matches(policy_actions: &[String], request_action: &str) -> bool {
for pa in policy_actions {
if pa == "*" || pa == "s3:*" {
return true;
}
if pa.eq_ignore_ascii_case(request_action) {
return true;
}
// Simple prefix wildcard: "s3:Get*" matches "s3:GetObject"
if let Some(prefix) = pa.strip_suffix('*') {
if request_action
.to_lowercase()
.starts_with(&prefix.to_lowercase())
{
return true;
}
}
}
false
}
/// Check if the resource matches. Supports wildcard patterns.
fn resource_matches(policy_resources: &[String], request_arn: &str, bucket: Option<&str>) -> bool {
for pr in policy_resources {
if pr == "*" {
return true;
}
if arn_pattern_matches(pr, request_arn) {
return true;
}
// Also check bucket-level ARN if the request is for an object
if let Some(b) = bucket {
let bucket_arn = format!("arn:aws:s3:::{}", b);
if arn_pattern_matches(pr, &bucket_arn) {
return true;
}
}
}
false
}
/// Simple ARN pattern matching with `*` and `?` wildcards.
fn arn_pattern_matches(pattern: &str, value: &str) -> bool {
// Handle trailing /* specifically: arn:aws:s3:::bucket/* matches arn:aws:s3:::bucket/anything
if pattern.ends_with("/*") {
let prefix = &pattern[..pattern.len() - 1]; // Remove trailing *
if value.starts_with(prefix) {
return true;
}
// Also match exact bucket without trailing /
let bucket_only = &pattern[..pattern.len() - 2];
if value == bucket_only {
return true;
}
}
simple_wildcard_match(pattern, value)
}
fn simple_wildcard_match(pattern: &str, value: &str) -> bool {
let pat_bytes = pattern.as_bytes();
let val_bytes = value.as_bytes();
let mut pi = 0;
let mut vi = 0;
let mut star_pi = usize::MAX;
let mut star_vi = 0;
while vi < val_bytes.len() {
if pi < pat_bytes.len() && (pat_bytes[pi] == b'?' || pat_bytes[pi] == val_bytes[vi]) {
pi += 1;
vi += 1;
} else if pi < pat_bytes.len() && pat_bytes[pi] == b'*' {
star_pi = pi;
star_vi = vi;
pi += 1;
} else if star_pi != usize::MAX {
pi = star_pi + 1;
star_vi += 1;
vi = star_vi;
} else {
return false;
}
}
while pi < pat_bytes.len() && pat_bytes[pi] == b'*' {
pi += 1;
}
pi == pat_bytes.len()
}
// ============================
// Policy validation
// ============================
const MAX_POLICY_SIZE: usize = 20 * 1024; // 20 KB
pub fn validate_policy(json: &str) -> Result<BucketPolicy, S3Error> {
if json.len() > MAX_POLICY_SIZE {
return Err(S3Error::malformed_policy("Policy exceeds maximum size of 20KB"));
}
let policy: BucketPolicy =
serde_json::from_str(json).map_err(|e| S3Error::malformed_policy(&e.to_string()))?;
if policy.version != "2012-10-17" {
return Err(S3Error::malformed_policy(
"Policy version must be \"2012-10-17\"",
));
}
if policy.statements.is_empty() {
return Err(S3Error::malformed_policy(
"Policy must contain at least one statement",
));
}
for (i, stmt) in policy.statements.iter().enumerate() {
if stmt.action.is_empty() {
return Err(S3Error::malformed_policy(&format!(
"Statement {} has no actions",
i
)));
}
for action in &stmt.action {
if action != "*" && !action.starts_with("s3:") {
return Err(S3Error::malformed_policy(&format!(
"Action \"{}\" must start with \"s3:\"",
action
)));
}
}
if stmt.resource.is_empty() {
return Err(S3Error::malformed_policy(&format!(
"Statement {} has no resources",
i
)));
}
for resource in &stmt.resource {
if resource != "*" && !resource.starts_with("arn:aws:s3:::") {
return Err(S3Error::malformed_policy(&format!(
"Resource \"{}\" must start with \"arn:aws:s3:::\"",
resource
)));
}
}
}
Ok(policy)
}
// ============================
// PolicyStore — in-memory cache + disk
// ============================
pub struct PolicyStore {
policies: RwLock<HashMap<String, BucketPolicy>>,
policies_dir: PathBuf,
}
impl PolicyStore {
pub fn new(policies_dir: PathBuf) -> Self {
Self {
policies: RwLock::new(HashMap::new()),
policies_dir,
}
}
/// Load all policies from disk into cache.
pub async fn load_from_disk(&self) -> anyhow::Result<()> {
let dir = &self.policies_dir;
if !dir.exists() {
return Ok(());
}
let mut entries = fs::read_dir(dir).await?;
let mut policies = HashMap::new();
while let Some(entry) = entries.next_entry().await? {
let name = entry.file_name().to_string_lossy().to_string();
if let Some(bucket) = name.strip_suffix(".policy.json") {
match fs::read_to_string(entry.path()).await {
Ok(json) => match serde_json::from_str::<BucketPolicy>(&json) {
Ok(policy) => {
tracing::info!("Loaded policy for bucket: {}", bucket);
policies.insert(bucket.to_string(), policy);
}
Err(e) => {
tracing::warn!("Failed to parse policy for {}: {}", bucket, e);
}
},
Err(e) => {
tracing::warn!("Failed to read policy file {}: {}", name, e);
}
}
}
}
let mut cache = self.policies.write().await;
*cache = policies;
Ok(())
}
/// Get a policy for a bucket.
pub async fn get_policy(&self, bucket: &str) -> Option<BucketPolicy> {
let cache = self.policies.read().await;
cache.get(bucket).cloned()
}
/// Store a policy for a bucket (atomic write + cache update).
pub async fn put_policy(&self, bucket: &str, policy: BucketPolicy) -> anyhow::Result<()> {
let json = serde_json::to_string_pretty(&policy)?;
// Atomic write: temp file + rename
let policy_path = self.policies_dir.join(format!("{}.policy.json", bucket));
let temp_path = self
.policies_dir
.join(format!("{}.policy.json.tmp", bucket));
fs::write(&temp_path, &json).await?;
fs::rename(&temp_path, &policy_path).await?;
// Update cache
let mut cache = self.policies.write().await;
cache.insert(bucket.to_string(), policy);
Ok(())
}
/// Delete a policy for a bucket.
pub async fn delete_policy(&self, bucket: &str) -> anyhow::Result<()> {
let policy_path = self.policies_dir.join(format!("{}.policy.json", bucket));
let _ = fs::remove_file(&policy_path).await;
let mut cache = self.policies.write().await;
cache.remove(bucket);
Ok(())
}
}

View File

@@ -1,6 +1,4 @@
use hyper::{Response, StatusCode}; use hyper::StatusCode;
use http_body_util::Full;
use bytes::Bytes;
#[derive(Debug, thiserror::Error)] #[derive(Debug, thiserror::Error)]
#[error("S3Error({code}): {message}")] #[error("S3Error({code}): {message}")]
@@ -51,20 +49,58 @@ impl S3Error {
Self::new("InvalidRequest", msg, StatusCode::BAD_REQUEST) Self::new("InvalidRequest", msg, StatusCode::BAD_REQUEST)
} }
pub fn signature_does_not_match() -> Self {
Self::new(
"SignatureDoesNotMatch",
"The request signature we calculated does not match the signature you provided.",
StatusCode::FORBIDDEN,
)
}
pub fn invalid_access_key_id() -> Self {
Self::new(
"InvalidAccessKeyId",
"The AWS Access Key Id you provided does not exist in our records.",
StatusCode::FORBIDDEN,
)
}
pub fn request_time_too_skewed() -> Self {
Self::new(
"RequestTimeTooSkewed",
"The difference between the request time and the current time is too large.",
StatusCode::FORBIDDEN,
)
}
pub fn authorization_header_malformed() -> Self {
Self::new(
"AuthorizationHeaderMalformed",
"The authorization header is malformed.",
StatusCode::BAD_REQUEST,
)
}
pub fn missing_security_header(msg: &str) -> Self {
Self::new("MissingSecurityHeader", msg, StatusCode::BAD_REQUEST)
}
pub fn no_such_bucket_policy() -> Self {
Self::new(
"NoSuchBucketPolicy",
"The bucket policy does not exist.",
StatusCode::NOT_FOUND,
)
}
pub fn malformed_policy(msg: &str) -> Self {
Self::new("MalformedPolicy", msg, StatusCode::BAD_REQUEST)
}
pub fn to_xml(&self) -> String { pub fn to_xml(&self) -> String {
format!( format!(
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>{}</Code><Message>{}</Message></Error>", "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>{}</Code><Message>{}</Message></Error>",
self.code, self.message self.code, self.message
) )
} }
pub fn to_response(&self, request_id: &str) -> Response<Full<Bytes>> {
let xml = self.to_xml();
Response::builder()
.status(self.status)
.header("content-type", "application/xml")
.header("x-amz-request-id", request_id)
.body(Full::new(Bytes::from(xml)))
.unwrap()
}
} }

View File

@@ -18,14 +18,16 @@ use tokio::sync::watch;
use tokio_util::io::ReaderStream; use tokio_util::io::ReaderStream;
use uuid::Uuid; use uuid::Uuid;
use crate::action::{self, RequestContext, S3Action};
use crate::auth::{self, AuthenticatedIdentity};
use crate::config::S3Config; use crate::config::S3Config;
use crate::policy::{self, PolicyDecision, PolicyStore};
use crate::s3_error::S3Error; use crate::s3_error::S3Error;
use crate::storage::FileStore; use crate::storage::FileStore;
use crate::xml_response; use crate::xml_response;
pub struct S3Server { pub struct S3Server {
store: Arc<FileStore>, store: Arc<FileStore>,
config: S3Config,
shutdown_tx: watch::Sender<bool>, shutdown_tx: watch::Sender<bool>,
server_handle: tokio::task::JoinHandle<()>, server_handle: tokio::task::JoinHandle<()>,
} }
@@ -41,6 +43,10 @@ impl S3Server {
store.initialize().await?; store.initialize().await?;
} }
// Initialize policy store
let policy_store = Arc::new(PolicyStore::new(store.policies_dir()));
policy_store.load_from_disk().await?;
let addr: SocketAddr = format!("{}:{}", config.address(), config.server.port) let addr: SocketAddr = format!("{}:{}", config.address(), config.server.port)
.parse()?; .parse()?;
@@ -49,6 +55,7 @@ impl S3Server {
let server_store = store.clone(); let server_store = store.clone();
let server_config = config.clone(); let server_config = config.clone();
let server_policy_store = policy_store.clone();
let server_handle = tokio::spawn(async move { let server_handle = tokio::spawn(async move {
loop { loop {
@@ -61,13 +68,15 @@ impl S3Server {
let io = TokioIo::new(stream); let io = TokioIo::new(stream);
let store = server_store.clone(); let store = server_store.clone();
let cfg = server_config.clone(); let cfg = server_config.clone();
let ps = server_policy_store.clone();
tokio::spawn(async move { tokio::spawn(async move {
let svc = service_fn(move |req: Request<Incoming>| { let svc = service_fn(move |req: Request<Incoming>| {
let store = store.clone(); let store = store.clone();
let cfg = cfg.clone(); let cfg = cfg.clone();
let ps = ps.clone();
async move { async move {
handle_request(req, store, cfg).await handle_request(req, store, cfg, ps).await
} }
}); });
@@ -100,7 +109,6 @@ impl S3Server {
Ok(Self { Ok(Self {
store, store,
config,
shutdown_tx, shutdown_tx,
server_handle, server_handle,
}) })
@@ -198,6 +206,7 @@ async fn handle_request(
req: Request<Incoming>, req: Request<Incoming>,
store: Arc<FileStore>, store: Arc<FileStore>,
config: S3Config, config: S3Config,
policy_store: Arc<PolicyStore>,
) -> Result<Response<BoxBody>, std::convert::Infallible> { ) -> Result<Response<BoxBody>, std::convert::Infallible> {
let request_id = Uuid::new_v4().to_string(); let request_id = Uuid::new_v4().to_string();
let method = req.method().clone(); let method = req.method().clone();
@@ -210,16 +219,41 @@ async fn handle_request(
return Ok(resp); return Ok(resp);
} }
// Auth check // Step 1: Resolve S3 action from request
let request_ctx = action::resolve_action(&req);
// Step 2: Auth + policy pipeline
if config.auth.enabled { if config.auth.enabled {
if let Err(e) = check_auth(&req, &config) { // Attempt authentication
tracing::warn!("Auth failed: {}", e.message); let identity = {
let has_auth_header = req
.headers()
.get("authorization")
.and_then(|v| v.to_str().ok())
.map(|s| !s.is_empty())
.unwrap_or(false);
if has_auth_header {
match auth::verify_request(&req, &config) {
Ok(id) => Some(id),
Err(e) => {
tracing::warn!("Auth failed: {}", e.message);
return Ok(s3_error_response(&e, &request_id));
}
}
} else {
None // Anonymous request
}
};
// Step 3: Authorization (policy evaluation)
if let Err(e) = authorize_request(&request_ctx, identity.as_ref(), &policy_store).await {
return Ok(s3_error_response(&e, &request_id)); return Ok(s3_error_response(&e, &request_id));
} }
} }
// Route and handle // Route and handle
let mut response = match route_request(req, store, &config, &request_id).await { let mut response = match route_request(req, store, &config, &request_id, &policy_store).await {
Ok(resp) => resp, Ok(resp) => resp,
Err(err) => { Err(err) => {
if let Some(s3err) = err.downcast_ref::<S3Error>() { if let Some(s3err) = err.downcast_ref::<S3Error>() {
@@ -249,6 +283,42 @@ async fn handle_request(
Ok(response) Ok(response)
} }
/// Authorize a request based on bucket policies and authentication state.
async fn authorize_request(
ctx: &RequestContext,
identity: Option<&AuthenticatedIdentity>,
policy_store: &PolicyStore,
) -> Result<(), S3Error> {
// ListAllMyBuckets requires authentication (no bucket to apply policy to)
if ctx.action == S3Action::ListAllMyBuckets {
if identity.is_none() {
return Err(S3Error::access_denied());
}
return Ok(());
}
// If there's a bucket, check its policy
if let Some(ref bucket) = ctx.bucket {
if let Some(bucket_policy) = policy_store.get_policy(bucket).await {
let decision = policy::evaluate_policy(&bucket_policy, ctx, identity);
match decision {
PolicyDecision::Deny => return Err(S3Error::access_denied()),
PolicyDecision::Allow => return Ok(()),
PolicyDecision::NoOpinion => {
// Fall through to default behavior
}
}
}
}
// Default: authenticated users get full access, anonymous denied
if identity.is_none() {
return Err(S3Error::access_denied());
}
Ok(())
}
// ============================ // ============================
// Routing // Routing
// ============================ // ============================
@@ -258,6 +328,7 @@ async fn route_request(
store: Arc<FileStore>, store: Arc<FileStore>,
_config: &S3Config, _config: &S3Config,
request_id: &str, request_id: &str,
policy_store: &Arc<PolicyStore>,
) -> Result<Response<BoxBody>> { ) -> Result<Response<BoxBody>> {
let method = req.method().clone(); let method = req.method().clone();
let path = req.uri().path().to_string(); let path = req.uri().path().to_string();
@@ -282,6 +353,17 @@ async fn route_request(
1 => { 1 => {
// Bucket level: /{bucket} // Bucket level: /{bucket}
let bucket = percent_decode(segments[0]); let bucket = percent_decode(segments[0]);
// Check for ?policy query parameter
if query.contains_key("policy") {
return match method {
Method::GET => handle_get_bucket_policy(policy_store, &bucket, request_id).await,
Method::PUT => handle_put_bucket_policy(req, &store, policy_store, &bucket, request_id).await,
Method::DELETE => handle_delete_bucket_policy(policy_store, &bucket, request_id).await,
_ => Ok(empty_response(StatusCode::METHOD_NOT_ALLOWED, request_id)),
};
}
match method { match method {
Method::GET => { Method::GET => {
if query.contains_key("uploads") { if query.contains_key("uploads") {
@@ -291,7 +373,7 @@ async fn route_request(
} }
} }
Method::PUT => handle_create_bucket(store, &bucket, request_id).await, Method::PUT => handle_create_bucket(store, &bucket, request_id).await,
Method::DELETE => handle_delete_bucket(store, &bucket, request_id).await, Method::DELETE => handle_delete_bucket(store, &bucket, request_id, policy_store).await,
Method::HEAD => handle_head_bucket(store, &bucket, request_id).await, Method::HEAD => handle_head_bucket(store, &bucket, request_id).await,
_ => Ok(empty_response(StatusCode::METHOD_NOT_ALLOWED, request_id)), _ => Ok(empty_response(StatusCode::METHOD_NOT_ALLOWED, request_id)),
} }
@@ -369,8 +451,11 @@ async fn handle_delete_bucket(
store: Arc<FileStore>, store: Arc<FileStore>,
bucket: &str, bucket: &str,
request_id: &str, request_id: &str,
policy_store: &Arc<PolicyStore>,
) -> Result<Response<BoxBody>> { ) -> Result<Response<BoxBody>> {
store.delete_bucket(bucket).await?; store.delete_bucket(bucket).await?;
// Clean up bucket policy on deletion
let _ = policy_store.delete_policy(bucket).await;
Ok(empty_response(StatusCode::NO_CONTENT, request_id)) Ok(empty_response(StatusCode::NO_CONTENT, request_id))
} }
@@ -577,6 +662,70 @@ async fn handle_copy_object(
Ok(xml_response(StatusCode::OK, xml, request_id)) Ok(xml_response(StatusCode::OK, xml, request_id))
} }
// ============================
// Policy handlers
// ============================
async fn handle_get_bucket_policy(
policy_store: &Arc<PolicyStore>,
bucket: &str,
request_id: &str,
) -> Result<Response<BoxBody>> {
match policy_store.get_policy(bucket).await {
Some(p) => {
let json = serde_json::to_string_pretty(&p)?;
let resp = Response::builder()
.status(StatusCode::OK)
.header("content-type", "application/json")
.header("x-amz-request-id", request_id)
.body(full_body(json))
.unwrap();
Ok(resp)
}
None => Err(S3Error::no_such_bucket_policy().into()),
}
}
async fn handle_put_bucket_policy(
req: Request<Incoming>,
store: &Arc<FileStore>,
policy_store: &Arc<PolicyStore>,
bucket: &str,
request_id: &str,
) -> Result<Response<BoxBody>> {
// Verify bucket exists
if !store.bucket_exists(bucket).await {
return Err(S3Error::no_such_bucket().into());
}
// Read body
let body_bytes = req.collect().await.map_err(|e| anyhow::anyhow!("Body error: {}", e))?.to_bytes();
let body_str = String::from_utf8_lossy(&body_bytes);
// Validate and parse
let validated_policy = policy::validate_policy(&body_str)?;
// Store
policy_store
.put_policy(bucket, validated_policy)
.await
.map_err(|e| S3Error::internal_error(&e.to_string()))?;
Ok(empty_response(StatusCode::NO_CONTENT, request_id))
}
async fn handle_delete_bucket_policy(
policy_store: &Arc<PolicyStore>,
bucket: &str,
request_id: &str,
) -> Result<Response<BoxBody>> {
policy_store
.delete_policy(bucket)
.await
.map_err(|e| S3Error::internal_error(&e.to_string()))?;
Ok(empty_response(StatusCode::NO_CONTENT, request_id))
}
// ============================ // ============================
// Multipart handlers // Multipart handlers
// ============================ // ============================
@@ -820,46 +969,3 @@ fn add_cors_headers(headers: &mut hyper::HeaderMap, config: &S3Config) {
); );
} }
} }
// ============================
// Auth
// ============================
fn check_auth(req: &Request<Incoming>, config: &S3Config) -> Result<(), S3Error> {
let auth_header = req
.headers()
.get("authorization")
.and_then(|v| v.to_str().ok())
.unwrap_or("");
if auth_header.is_empty() {
return Err(S3Error::access_denied());
}
// Extract access key from AWS v2 or v4 signature
let access_key = if auth_header.starts_with("AWS4-HMAC-SHA256") {
// v4: AWS4-HMAC-SHA256 Credential=KEY/date/region/s3/aws4_request, ...
auth_header
.split("Credential=")
.nth(1)
.and_then(|s| s.split('/').next())
} else if auth_header.starts_with("AWS ") {
// v2: AWS KEY:signature
auth_header
.strip_prefix("AWS ")
.and_then(|s| s.split(':').next())
} else {
None
};
let access_key = access_key.unwrap_or("");
// Check against configured credentials
for cred in &config.auth.credentials {
if cred.access_key_id == access_key {
return Ok(());
}
}
Err(S3Error::access_denied())
}

View File

@@ -17,12 +17,10 @@ use crate::s3_error::S3Error;
// ============================ // ============================
pub struct PutResult { pub struct PutResult {
pub size: u64,
pub md5: String, pub md5: String,
} }
pub struct GetResult { pub struct GetResult {
pub key: String,
pub size: u64, pub size: u64,
pub last_modified: DateTime<Utc>, pub last_modified: DateTime<Utc>,
pub md5: String, pub md5: String,
@@ -32,7 +30,6 @@ pub struct GetResult {
} }
pub struct HeadResult { pub struct HeadResult {
pub key: String,
pub size: u64, pub size: u64,
pub last_modified: DateTime<Utc>, pub last_modified: DateTime<Utc>,
pub md5: String, pub md5: String,
@@ -40,7 +37,6 @@ pub struct HeadResult {
} }
pub struct CopyResult { pub struct CopyResult {
pub size: u64,
pub md5: String, pub md5: String,
pub last_modified: DateTime<Utc>, pub last_modified: DateTime<Utc>,
} }
@@ -69,14 +65,12 @@ pub struct BucketInfo {
pub struct MultipartUploadInfo { pub struct MultipartUploadInfo {
pub upload_id: String, pub upload_id: String,
pub bucket: String,
pub key: String, pub key: String,
pub initiated: DateTime<Utc>, pub initiated: DateTime<Utc>,
} }
pub struct CompleteMultipartResult { pub struct CompleteMultipartResult {
pub etag: String, pub etag: String,
pub size: u64,
} }
// ============================ // ============================
@@ -118,14 +112,20 @@ impl FileStore {
pub async fn initialize(&self) -> Result<()> { pub async fn initialize(&self) -> Result<()> {
fs::create_dir_all(&self.root_dir).await?; fs::create_dir_all(&self.root_dir).await?;
fs::create_dir_all(self.policies_dir()).await?;
Ok(()) Ok(())
} }
pub fn policies_dir(&self) -> PathBuf {
self.root_dir.join(".policies")
}
pub async fn reset(&self) -> Result<()> { pub async fn reset(&self) -> Result<()> {
if self.root_dir.exists() { if self.root_dir.exists() {
fs::remove_dir_all(&self.root_dir).await?; fs::remove_dir_all(&self.root_dir).await?;
} }
fs::create_dir_all(&self.root_dir).await?; fs::create_dir_all(&self.root_dir).await?;
fs::create_dir_all(self.policies_dir()).await?;
Ok(()) Ok(())
} }
@@ -210,7 +210,6 @@ impl FileStore {
let file = fs::File::create(&object_path).await?; let file = fs::File::create(&object_path).await?;
let mut writer = BufWriter::new(file); let mut writer = BufWriter::new(file);
let mut hasher = Md5::new(); let mut hasher = Md5::new();
let mut total_size: u64 = 0;
// Stream body frames directly to file // Stream body frames directly to file
let mut body = body; let mut body = body;
@@ -219,7 +218,6 @@ impl FileStore {
Some(Ok(frame)) => { Some(Ok(frame)) => {
if let Ok(data) = frame.into_data() { if let Ok(data) = frame.into_data() {
hasher.update(&data); hasher.update(&data);
total_size += data.len() as u64;
writer.write_all(&data).await?; writer.write_all(&data).await?;
} }
} }
@@ -245,44 +243,6 @@ impl FileStore {
fs::write(&metadata_path, metadata_json).await?; fs::write(&metadata_path, metadata_json).await?;
Ok(PutResult { Ok(PutResult {
size: total_size,
md5: md5_hex,
})
}
pub async fn put_object_bytes(
&self,
bucket: &str,
key: &str,
data: &[u8],
metadata: HashMap<String, String>,
) -> Result<PutResult> {
if !self.bucket_exists(bucket).await {
return Err(S3Error::no_such_bucket().into());
}
let object_path = self.object_path(bucket, key);
if let Some(parent) = object_path.parent() {
fs::create_dir_all(parent).await?;
}
let mut hasher = Md5::new();
hasher.update(data);
let md5_hex = format!("{:x}", hasher.finalize());
fs::write(&object_path, data).await?;
// Write MD5 sidecar
let md5_path = format!("{}.md5", object_path.display());
fs::write(&md5_path, &md5_hex).await?;
// Write metadata sidecar
let metadata_path = format!("{}.metadata.json", object_path.display());
let metadata_json = serde_json::to_string_pretty(&metadata)?;
fs::write(&metadata_path, metadata_json).await?;
Ok(PutResult {
size: data.len() as u64,
md5: md5_hex, md5: md5_hex,
}) })
} }
@@ -316,7 +276,6 @@ impl FileStore {
}; };
Ok(GetResult { Ok(GetResult {
key: key.to_string(),
size, size,
last_modified, last_modified,
md5, md5,
@@ -342,7 +301,6 @@ impl FileStore {
let metadata = self.read_metadata(&object_path).await; let metadata = self.read_metadata(&object_path).await;
Ok(HeadResult { Ok(HeadResult {
key: key.to_string(),
size, size,
last_modified, last_modified,
md5, md5,
@@ -429,7 +387,6 @@ impl FileStore {
let last_modified: DateTime<Utc> = file_meta.modified()?.into(); let last_modified: DateTime<Utc> = file_meta.modified()?.into();
Ok(CopyResult { Ok(CopyResult {
size: file_meta.len(),
md5, md5,
last_modified, last_modified,
}) })
@@ -662,7 +619,6 @@ impl FileStore {
let dest_file = fs::File::create(&object_path).await?; let dest_file = fs::File::create(&object_path).await?;
let mut writer = BufWriter::new(dest_file); let mut writer = BufWriter::new(dest_file);
let mut hasher = Md5::new(); let mut hasher = Md5::new();
let mut total_size: u64 = 0;
for (part_number, _etag) in parts { for (part_number, _etag) in parts {
let part_path = upload_dir.join(format!("part-{}", part_number)); let part_path = upload_dir.join(format!("part-{}", part_number));
@@ -679,7 +635,6 @@ impl FileStore {
} }
hasher.update(&buf[..n]); hasher.update(&buf[..n]);
writer.write_all(&buf[..n]).await?; writer.write_all(&buf[..n]).await?;
total_size += n as u64;
} }
} }
@@ -702,7 +657,6 @@ impl FileStore {
Ok(CompleteMultipartResult { Ok(CompleteMultipartResult {
etag, etag,
size: total_size,
}) })
} }
@@ -742,7 +696,6 @@ impl FileStore {
uploads.push(MultipartUploadInfo { uploads.push(MultipartUploadInfo {
upload_id: meta.upload_id, upload_id: meta.upload_id,
bucket: meta.bucket,
key: meta.key, key: meta.key,
initiated, initiated,
}); });

View File

@@ -132,15 +132,6 @@ pub fn list_objects_v2_xml(bucket: &str, result: &ListObjectsResult) -> String {
xml xml
} }
pub fn error_xml(code: &str, message: &str) -> String {
format!(
"{}\n<Error><Code>{}</Code><Message>{}</Message></Error>",
XML_DECL,
xml_escape(code),
xml_escape(message)
)
}
pub fn copy_object_result_xml(etag: &str, last_modified: &str) -> String { pub fn copy_object_result_xml(etag: &str, last_modified: &str) -> String {
format!( format!(
"{}\n<CopyObjectResult>\ "{}\n<CopyObjectResult>\

301
test/test.auth.node.ts Normal file
View File

@@ -0,0 +1,301 @@
import { expect, tap } from '@git.zone/tstest/tapbundle';
import {
S3Client,
CreateBucketCommand,
ListBucketsCommand,
PutObjectCommand,
GetObjectCommand,
DeleteObjectCommand,
DeleteBucketCommand,
PutBucketPolicyCommand,
GetBucketPolicyCommand,
DeleteBucketPolicyCommand,
} from '@aws-sdk/client-s3';
import { Readable } from 'stream';
import * as smarts3 from '../ts/index.js';
let testSmarts3Instance: smarts3.Smarts3;
let authClient: S3Client;
let wrongClient: S3Client;
const TEST_PORT = 3344;
const ACCESS_KEY = 'TESTAKID';
const SECRET_KEY = 'TESTSECRETKEY123';
async function streamToString(stream: Readable): Promise<string> {
const chunks: Buffer[] = [];
return new Promise((resolve, reject) => {
stream.on('data', (chunk) => chunks.push(Buffer.from(chunk)));
stream.on('error', reject);
stream.on('end', () => resolve(Buffer.concat(chunks).toString('utf8')));
});
}
// ============================
// Server setup
// ============================
tap.test('should start S3 server with auth enabled', async () => {
testSmarts3Instance = await smarts3.Smarts3.createAndStart({
server: {
port: TEST_PORT,
silent: true,
region: 'us-east-1',
},
storage: {
cleanSlate: true,
},
auth: {
enabled: true,
credentials: [
{
accessKeyId: ACCESS_KEY,
secretAccessKey: SECRET_KEY,
},
],
},
});
// Authenticated client with correct credentials
authClient = new S3Client({
endpoint: `http://localhost:${TEST_PORT}`,
region: 'us-east-1',
credentials: {
accessKeyId: ACCESS_KEY,
secretAccessKey: SECRET_KEY,
},
forcePathStyle: true,
});
// Client with wrong credentials
wrongClient = new S3Client({
endpoint: `http://localhost:${TEST_PORT}`,
region: 'us-east-1',
credentials: {
accessKeyId: 'WRONGKEY',
secretAccessKey: 'WRONGSECRET',
},
forcePathStyle: true,
});
});
// ============================
// Authenticated CRUD
// ============================
tap.test('authenticated: should list buckets', async () => {
const response = await authClient.send(new ListBucketsCommand({}));
expect(response.$metadata.httpStatusCode).toEqual(200);
expect(Array.isArray(response.Buckets)).toEqual(true);
});
tap.test('authenticated: should create a bucket', async () => {
const response = await authClient.send(new CreateBucketCommand({ Bucket: 'auth-test-bucket' }));
expect(response.$metadata.httpStatusCode).toEqual(200);
});
tap.test('authenticated: should upload an object', async () => {
const response = await authClient.send(
new PutObjectCommand({
Bucket: 'auth-test-bucket',
Key: 'hello.txt',
Body: 'Hello authenticated world!',
ContentType: 'text/plain',
}),
);
expect(response.$metadata.httpStatusCode).toEqual(200);
});
tap.test('authenticated: should download the object', async () => {
const response = await authClient.send(
new GetObjectCommand({
Bucket: 'auth-test-bucket',
Key: 'hello.txt',
}),
);
expect(response.$metadata.httpStatusCode).toEqual(200);
const content = await streamToString(response.Body as Readable);
expect(content).toEqual('Hello authenticated world!');
});
// ============================
// Wrong credentials → 403
// ============================
tap.test('wrong credentials: should fail to list buckets', async () => {
await expect(wrongClient.send(new ListBucketsCommand({}))).rejects.toThrow();
});
tap.test('wrong credentials: should fail to get object', async () => {
await expect(
wrongClient.send(
new GetObjectCommand({
Bucket: 'auth-test-bucket',
Key: 'hello.txt',
}),
),
).rejects.toThrow();
});
// ============================
// Anonymous → 403 (no policy yet)
// ============================
tap.test('anonymous: should fail to list buckets', async () => {
const resp = await fetch(`http://localhost:${TEST_PORT}/`);
expect(resp.status).toEqual(403);
});
tap.test('anonymous: should fail to get object (no policy)', async () => {
const resp = await fetch(`http://localhost:${TEST_PORT}/auth-test-bucket/hello.txt`);
expect(resp.status).toEqual(403);
});
// ============================
// Bucket policy: public read
// ============================
tap.test('should PUT a public-read bucket policy', async () => {
const policy = {
Version: '2012-10-17',
Statement: [
{
Sid: 'PublicRead',
Effect: 'Allow',
Principal: '*',
Action: ['s3:GetObject'],
Resource: [`arn:aws:s3:::auth-test-bucket/*`],
},
],
};
const response = await authClient.send(
new PutBucketPolicyCommand({
Bucket: 'auth-test-bucket',
Policy: JSON.stringify(policy),
}),
);
expect(response.$metadata.httpStatusCode).toEqual(204);
});
tap.test('should GET the bucket policy', async () => {
const response = await authClient.send(
new GetBucketPolicyCommand({
Bucket: 'auth-test-bucket',
}),
);
expect(response.$metadata.httpStatusCode).toEqual(200);
const policy = JSON.parse(response.Policy!);
expect(policy.Statement[0].Sid).toEqual('PublicRead');
});
tap.test('anonymous: should GET object after public-read policy', async () => {
const resp = await fetch(`http://localhost:${TEST_PORT}/auth-test-bucket/hello.txt`);
expect(resp.status).toEqual(200);
const content = await resp.text();
expect(content).toEqual('Hello authenticated world!');
});
tap.test('anonymous: should still fail to PUT object (policy only allows GET)', async () => {
const resp = await fetch(`http://localhost:${TEST_PORT}/auth-test-bucket/anon-file.txt`, {
method: 'PUT',
body: 'should fail',
});
expect(resp.status).toEqual(403);
});
// ============================
// Deny policy
// ============================
tap.test('should PUT a deny policy that blocks authenticated delete', async () => {
const policy = {
Version: '2012-10-17',
Statement: [
{
Sid: 'PublicRead',
Effect: 'Allow',
Principal: '*',
Action: ['s3:GetObject'],
Resource: [`arn:aws:s3:::auth-test-bucket/*`],
},
{
Sid: 'DenyDelete',
Effect: 'Deny',
Principal: '*',
Action: ['s3:DeleteObject'],
Resource: [`arn:aws:s3:::auth-test-bucket/*`],
},
],
};
const response = await authClient.send(
new PutBucketPolicyCommand({
Bucket: 'auth-test-bucket',
Policy: JSON.stringify(policy),
}),
);
expect(response.$metadata.httpStatusCode).toEqual(204);
});
tap.test('authenticated: should be denied delete by policy', async () => {
await expect(
authClient.send(
new DeleteObjectCommand({
Bucket: 'auth-test-bucket',
Key: 'hello.txt',
}),
),
).rejects.toThrow();
});
// ============================
// DELETE bucket policy
// ============================
tap.test('should DELETE the bucket policy', async () => {
const response = await authClient.send(
new DeleteBucketPolicyCommand({
Bucket: 'auth-test-bucket',
}),
);
expect(response.$metadata.httpStatusCode).toEqual(204);
});
tap.test('should GET policy → 404 after deletion', async () => {
await expect(
authClient.send(
new GetBucketPolicyCommand({
Bucket: 'auth-test-bucket',
}),
),
).rejects.toThrow();
});
// ============================
// Cleanup
// ============================
tap.test('authenticated: delete object after policy removed', async () => {
const response = await authClient.send(
new DeleteObjectCommand({
Bucket: 'auth-test-bucket',
Key: 'hello.txt',
}),
);
expect(response.$metadata.httpStatusCode).toEqual(204);
});
tap.test('authenticated: delete the bucket', async () => {
const response = await authClient.send(
new DeleteBucketCommand({ Bucket: 'auth-test-bucket' }),
);
expect(response.$metadata.httpStatusCode).toEqual(204);
});
tap.test('should stop the S3 server', async () => {
await testSmarts3Instance.stop();
});
export default tap.start();

View File

@@ -0,0 +1,335 @@
import { expect, tap } from '@git.zone/tstest/tapbundle';
import {
S3Client,
CreateBucketCommand,
DeleteBucketCommand,
ListBucketsCommand,
ListObjectsV2Command,
PutObjectCommand,
GetObjectCommand,
DeleteObjectCommand,
CopyObjectCommand,
HeadBucketCommand,
PutBucketPolicyCommand,
GetBucketPolicyCommand,
DeleteBucketPolicyCommand,
} from '@aws-sdk/client-s3';
import * as smarts3 from '../ts/index.js';
let testSmarts3Instance: smarts3.Smarts3;
let authClient: S3Client;
const TEST_PORT = 3347;
const ACCESS_KEY = 'TESTAKID';
const SECRET_KEY = 'TESTSECRETKEY123';
const BUCKET = 'actions-bucket';
const BASE_URL = `http://localhost:${TEST_PORT}`;
async function putPolicy(statements: any[]) {
await authClient.send(
new PutBucketPolicyCommand({
Bucket: BUCKET,
Policy: JSON.stringify({ Version: '2012-10-17', Statement: statements }),
})
);
}
async function clearPolicy() {
await authClient.send(new DeleteBucketPolicyCommand({ Bucket: BUCKET }));
}
function denyStatement(action: string) {
return {
Sid: `Deny_${action.replace(':', '_')}`,
Effect: 'Deny' as const,
Principal: '*',
Action: action,
Resource: [
`arn:aws:s3:::${BUCKET}`,
`arn:aws:s3:::${BUCKET}/*`,
],
};
}
// ============================
// Server setup
// ============================
tap.test('setup: start server, create bucket, upload object', async () => {
testSmarts3Instance = await smarts3.Smarts3.createAndStart({
server: { port: TEST_PORT, silent: true, region: 'us-east-1' },
storage: { cleanSlate: true },
auth: {
enabled: true,
credentials: [{ accessKeyId: ACCESS_KEY, secretAccessKey: SECRET_KEY }],
},
});
authClient = new S3Client({
endpoint: BASE_URL,
region: 'us-east-1',
credentials: { accessKeyId: ACCESS_KEY, secretAccessKey: SECRET_KEY },
forcePathStyle: true,
});
await authClient.send(new CreateBucketCommand({ Bucket: BUCKET }));
await authClient.send(
new PutObjectCommand({
Bucket: BUCKET,
Key: 'obj.txt',
Body: 'test content for actions',
ContentType: 'text/plain',
})
);
});
// ============================
// Per-action deny enforcement
// ============================
tap.test('Deny s3:ListBucket → authenticated ListObjects fails', async () => {
await putPolicy([denyStatement('s3:ListBucket')]);
await expect(
authClient.send(new ListObjectsV2Command({ Bucket: BUCKET }))
).rejects.toThrow();
await clearPolicy();
});
tap.test('Deny s3:CreateBucket → authenticated CreateBucket on new bucket fails', async () => {
// We need to create a policy on the target bucket, but the target doesn't exist yet.
// Instead, we use a different approach: deny on existing bucket and test HeadBucket works
// but for CreateBucket, use fetch to target a new bucket name with the deny check.
// Actually, CreateBucket has no bucket policy to evaluate against (the bucket doesn't exist yet).
// The deny would need to be on the bucket being created.
// Since the bucket doesn't exist, there's no policy to load — so CreateBucket can't be denied via policy.
// This is expected AWS behavior. Skip this test and note it.
// Verify CreateBucket still works (no policy can deny it since bucket doesn't exist yet)
await authClient.send(new CreateBucketCommand({ Bucket: 'new-test-bucket' }));
await authClient.send(new DeleteBucketCommand({ Bucket: 'new-test-bucket' }));
});
tap.test('Deny s3:DeleteBucket → authenticated DeleteBucket fails', async () => {
await putPolicy([denyStatement('s3:DeleteBucket')]);
await expect(
authClient.send(new DeleteBucketCommand({ Bucket: BUCKET }))
).rejects.toThrow();
await clearPolicy();
});
tap.test('Deny s3:GetObject → authenticated GetObject fails', async () => {
await putPolicy([denyStatement('s3:GetObject')]);
await expect(
authClient.send(new GetObjectCommand({ Bucket: BUCKET, Key: 'obj.txt' }))
).rejects.toThrow();
await clearPolicy();
});
tap.test('Deny s3:PutObject → authenticated PutObject fails', async () => {
await putPolicy([denyStatement('s3:PutObject')]);
await expect(
authClient.send(
new PutObjectCommand({
Bucket: BUCKET,
Key: 'new-obj.txt',
Body: 'should fail',
})
)
).rejects.toThrow();
await clearPolicy();
});
tap.test('Deny s3:DeleteObject → authenticated DeleteObject fails', async () => {
await putPolicy([denyStatement('s3:DeleteObject')]);
await expect(
authClient.send(new DeleteObjectCommand({ Bucket: BUCKET, Key: 'obj.txt' }))
).rejects.toThrow();
await clearPolicy();
});
tap.test('Deny s3:PutObject → authenticated CopyObject fails (maps to s3:PutObject)', async () => {
await putPolicy([denyStatement('s3:PutObject')]);
await expect(
authClient.send(
new CopyObjectCommand({
Bucket: BUCKET,
Key: 'obj-copy.txt',
CopySource: `${BUCKET}/obj.txt`,
})
)
).rejects.toThrow();
await clearPolicy();
});
tap.test('Deny s3:GetBucketPolicy → authenticated GetBucketPolicy fails', async () => {
// First put a policy that denies GetBucketPolicy
// We need to be careful: put the deny policy, then try to get it
await putPolicy([denyStatement('s3:GetBucketPolicy')]);
await expect(
authClient.send(new GetBucketPolicyCommand({ Bucket: BUCKET }))
).rejects.toThrow();
// Clear using direct delete (which isn't denied)
await clearPolicy();
});
tap.test('Deny s3:PutBucketPolicy → authenticated PutBucketPolicy fails (for second policy)', async () => {
// First put a policy that denies PutBucketPolicy
await putPolicy([denyStatement('s3:PutBucketPolicy')]);
// Now try to put another policy — should fail
await expect(
authClient.send(
new PutBucketPolicyCommand({
Bucket: BUCKET,
Policy: JSON.stringify({
Version: '2012-10-17',
Statement: [
{
Sid: 'SomeOtherPolicy',
Effect: 'Allow',
Principal: '*',
Action: 's3:GetObject',
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
],
}),
})
)
).rejects.toThrow();
await clearPolicy();
});
tap.test('Deny s3:DeleteBucketPolicy → authenticated DeleteBucketPolicy fails', async () => {
await putPolicy([denyStatement('s3:DeleteBucketPolicy')]);
await expect(
authClient.send(new DeleteBucketPolicyCommand({ Bucket: BUCKET }))
).rejects.toThrow();
// We need another way to clean up — use fetch with auth to bypass? No, the deny is on all principals.
// Actually, we can't clear the policy via SDK since delete is denied.
// The server still denies it. We need to stop and restart or use a different mechanism.
// For test cleanup, just stop the server at end and it will be wiped with cleanSlate on next start.
});
tap.test('Recovery: remove deny policy → authenticated operations resume working', async () => {
// The previous test left a deny policy on DeleteBucketPolicy.
// But we can work around it by stopping/restarting or if the deny is still in place.
// Actually, we denied s3:DeleteBucketPolicy but NOT s3:PutBucketPolicy.
// So we can overwrite the policy with an empty-ish one, then delete.
await authClient.send(
new PutBucketPolicyCommand({
Bucket: BUCKET,
Policy: JSON.stringify({
Version: '2012-10-17',
Statement: [
{
Sid: 'AllowAll',
Effect: 'Allow',
Principal: '*',
Action: 's3:*',
Resource: [`arn:aws:s3:::${BUCKET}`, `arn:aws:s3:::${BUCKET}/*`],
},
],
}),
})
);
// Now all operations should work again
const getResp = await authClient.send(
new GetObjectCommand({ Bucket: BUCKET, Key: 'obj.txt' })
);
expect(getResp.$metadata.httpStatusCode).toEqual(200);
const listResp = await authClient.send(
new ListObjectsV2Command({ Bucket: BUCKET })
);
expect(listResp.$metadata.httpStatusCode).toEqual(200);
await clearPolicy();
});
// ============================
// Special cases
// ============================
tap.test('ListAllMyBuckets always requires auth → anonymous fetch to / returns 403', async () => {
const resp = await fetch(`${BASE_URL}/`);
expect(resp.status).toEqual(403);
});
tap.test('Auth disabled mode → anonymous full access works', async () => {
// Start a second server with auth disabled
const noAuthInstance = await smarts3.Smarts3.createAndStart({
server: { port: 3348, silent: true, region: 'us-east-1' },
storage: { cleanSlate: true },
auth: { enabled: false, credentials: [] },
});
// Anonymous operations should all work
const listResp = await fetch('http://localhost:3348/');
expect(listResp.status).toEqual(200);
// Create bucket via fetch
const createResp = await fetch('http://localhost:3348/anon-bucket', { method: 'PUT' });
expect(createResp.status).toEqual(200);
// Put object
const putResp = await fetch('http://localhost:3348/anon-bucket/file.txt', {
method: 'PUT',
body: 'hello anon',
});
expect(putResp.status).toEqual(200);
// Get object
const getResp = await fetch('http://localhost:3348/anon-bucket/file.txt');
expect(getResp.status).toEqual(200);
const text = await getResp.text();
expect(text).toEqual('hello anon');
// Delete object
const delObjResp = await fetch('http://localhost:3348/anon-bucket/file.txt', { method: 'DELETE' });
expect(delObjResp.status).toEqual(204);
// Delete bucket
const delBucketResp = await fetch('http://localhost:3348/anon-bucket', { method: 'DELETE' });
expect(delBucketResp.status).toEqual(204);
await noAuthInstance.stop();
});
// ============================
// Teardown
// ============================
tap.test('teardown: clean up and stop server', async () => {
// Clean up any remaining objects
try {
await authClient.send(new DeleteObjectCommand({ Bucket: BUCKET, Key: 'obj.txt' }));
} catch {
// May already be deleted
}
try {
await authClient.send(new DeleteBucketCommand({ Bucket: BUCKET }));
} catch {
// May already be deleted
}
await testSmarts3Instance.stop();
});
export default tap.start();

View File

@@ -0,0 +1,252 @@
import { expect, tap } from '@git.zone/tstest/tapbundle';
import {
S3Client,
CreateBucketCommand,
DeleteBucketCommand,
PutBucketPolicyCommand,
GetBucketPolicyCommand,
DeleteBucketPolicyCommand,
} from '@aws-sdk/client-s3';
import * as smarts3 from '../ts/index.js';
let testSmarts3Instance: smarts3.Smarts3;
let authClient: S3Client;
const TEST_PORT = 3345;
const ACCESS_KEY = 'TESTAKID';
const SECRET_KEY = 'TESTSECRETKEY123';
const BUCKET = 'policy-crud-bucket';
function makePolicy(statements: any[]) {
return JSON.stringify({ Version: '2012-10-17', Statement: statements });
}
const validStatement = {
Sid: 'Test1',
Effect: 'Allow',
Principal: '*',
Action: ['s3:GetObject'],
Resource: [`arn:aws:s3:::${BUCKET}/*`],
};
// ============================
// Server setup
// ============================
tap.test('setup: start S3 server with auth enabled', async () => {
testSmarts3Instance = await smarts3.Smarts3.createAndStart({
server: { port: TEST_PORT, silent: true, region: 'us-east-1' },
storage: { cleanSlate: true },
auth: {
enabled: true,
credentials: [{ accessKeyId: ACCESS_KEY, secretAccessKey: SECRET_KEY }],
},
});
authClient = new S3Client({
endpoint: `http://localhost:${TEST_PORT}`,
region: 'us-east-1',
credentials: { accessKeyId: ACCESS_KEY, secretAccessKey: SECRET_KEY },
forcePathStyle: true,
});
});
tap.test('setup: create bucket', async () => {
await authClient.send(new CreateBucketCommand({ Bucket: BUCKET }));
});
// ============================
// CRUD tests
// ============================
tap.test('GET policy on bucket with no policy → throws (NoSuchBucketPolicy)', async () => {
await expect(
authClient.send(new GetBucketPolicyCommand({ Bucket: BUCKET }))
).rejects.toThrow();
});
tap.test('PUT valid policy → 204', async () => {
const response = await authClient.send(
new PutBucketPolicyCommand({
Bucket: BUCKET,
Policy: makePolicy([validStatement]),
})
);
expect(response.$metadata.httpStatusCode).toEqual(204);
});
tap.test('GET policy back → returns matching JSON', async () => {
const response = await authClient.send(
new GetBucketPolicyCommand({ Bucket: BUCKET })
);
expect(response.$metadata.httpStatusCode).toEqual(200);
const policy = JSON.parse(response.Policy!);
expect(policy.Version).toEqual('2012-10-17');
expect(policy.Statement[0].Sid).toEqual('Test1');
expect(policy.Statement[0].Effect).toEqual('Allow');
});
tap.test('PUT updated policy (overwrite) → 204, GET returns new version', async () => {
const updatedStatement = {
Sid: 'Updated',
Effect: 'Deny',
Principal: '*',
Action: ['s3:DeleteObject'],
Resource: [`arn:aws:s3:::${BUCKET}/*`],
};
const putResp = await authClient.send(
new PutBucketPolicyCommand({
Bucket: BUCKET,
Policy: makePolicy([updatedStatement]),
})
);
expect(putResp.$metadata.httpStatusCode).toEqual(204);
const getResp = await authClient.send(
new GetBucketPolicyCommand({ Bucket: BUCKET })
);
const policy = JSON.parse(getResp.Policy!);
expect(policy.Statement[0].Sid).toEqual('Updated');
expect(policy.Statement[0].Effect).toEqual('Deny');
});
tap.test('DELETE policy → 204', async () => {
const response = await authClient.send(
new DeleteBucketPolicyCommand({ Bucket: BUCKET })
);
expect(response.$metadata.httpStatusCode).toEqual(204);
});
tap.test('DELETE policy again (idempotent) → 204', async () => {
const response = await authClient.send(
new DeleteBucketPolicyCommand({ Bucket: BUCKET })
);
expect(response.$metadata.httpStatusCode).toEqual(204);
});
tap.test('GET policy after delete → throws', async () => {
await expect(
authClient.send(new GetBucketPolicyCommand({ Bucket: BUCKET }))
).rejects.toThrow();
});
tap.test('PUT policy on non-existent bucket → throws (NoSuchBucket)', async () => {
await expect(
authClient.send(
new PutBucketPolicyCommand({
Bucket: 'nonexistent-bucket-xyz',
Policy: makePolicy([validStatement]),
})
)
).rejects.toThrow();
});
tap.test('PUT invalid JSON → throws (MalformedPolicy)', async () => {
await expect(
authClient.send(
new PutBucketPolicyCommand({
Bucket: BUCKET,
Policy: '{not valid json!!!',
})
)
).rejects.toThrow();
});
tap.test('PUT policy with wrong version → throws (MalformedPolicy)', async () => {
await expect(
authClient.send(
new PutBucketPolicyCommand({
Bucket: BUCKET,
Policy: JSON.stringify({
Version: '2023-01-01',
Statement: [validStatement],
}),
})
)
).rejects.toThrow();
});
tap.test('PUT policy with empty statements array → throws (MalformedPolicy)', async () => {
await expect(
authClient.send(
new PutBucketPolicyCommand({
Bucket: BUCKET,
Policy: JSON.stringify({
Version: '2012-10-17',
Statement: [],
}),
})
)
).rejects.toThrow();
});
tap.test('PUT policy with action missing s3: prefix → throws (MalformedPolicy)', async () => {
await expect(
authClient.send(
new PutBucketPolicyCommand({
Bucket: BUCKET,
Policy: makePolicy([
{
Sid: 'BadAction',
Effect: 'Allow',
Principal: '*',
Action: ['GetObject'],
Resource: [`arn:aws:s3:::${BUCKET}/*`],
},
]),
})
)
).rejects.toThrow();
});
tap.test('PUT policy with resource missing arn:aws:s3::: prefix → throws (MalformedPolicy)', async () => {
await expect(
authClient.send(
new PutBucketPolicyCommand({
Bucket: BUCKET,
Policy: makePolicy([
{
Sid: 'BadResource',
Effect: 'Allow',
Principal: '*',
Action: ['s3:GetObject'],
Resource: ['policy-crud-bucket/*'],
},
]),
})
)
).rejects.toThrow();
});
tap.test('Bucket deletion cleans up associated policy', async () => {
// PUT a policy
await authClient.send(
new PutBucketPolicyCommand({
Bucket: BUCKET,
Policy: makePolicy([validStatement]),
})
);
// Delete the bucket
await authClient.send(new DeleteBucketCommand({ Bucket: BUCKET }));
// Re-create the bucket
await authClient.send(new CreateBucketCommand({ Bucket: BUCKET }));
// GET policy should now be gone
await expect(
authClient.send(new GetBucketPolicyCommand({ Bucket: BUCKET }))
).rejects.toThrow();
});
// ============================
// Teardown
// ============================
tap.test('teardown: delete bucket and stop server', async () => {
await authClient.send(new DeleteBucketCommand({ Bucket: BUCKET }));
await testSmarts3Instance.stop();
});
export default tap.start();

View File

@@ -0,0 +1,517 @@
import { expect, tap } from '@git.zone/tstest/tapbundle';
import {
S3Client,
CreateBucketCommand,
DeleteBucketCommand,
PutObjectCommand,
GetObjectCommand,
DeleteObjectCommand,
PutBucketPolicyCommand,
DeleteBucketPolicyCommand,
} from '@aws-sdk/client-s3';
import { Readable } from 'stream';
import * as smarts3 from '../ts/index.js';
let testSmarts3Instance: smarts3.Smarts3;
let authClient: S3Client;
const TEST_PORT = 3346;
const ACCESS_KEY = 'TESTAKID';
const SECRET_KEY = 'TESTSECRETKEY123';
const BUCKET = 'eval-bucket';
const BASE_URL = `http://localhost:${TEST_PORT}`;
async function streamToString(stream: Readable): Promise<string> {
const chunks: Buffer[] = [];
return new Promise((resolve, reject) => {
stream.on('data', (chunk) => chunks.push(Buffer.from(chunk)));
stream.on('error', reject);
stream.on('end', () => resolve(Buffer.concat(chunks).toString('utf8')));
});
}
async function putPolicy(statements: any[]) {
await authClient.send(
new PutBucketPolicyCommand({
Bucket: BUCKET,
Policy: JSON.stringify({ Version: '2012-10-17', Statement: statements }),
})
);
}
async function clearPolicy() {
await authClient.send(new DeleteBucketPolicyCommand({ Bucket: BUCKET }));
}
// ============================
// Server setup
// ============================
tap.test('setup: start server, create bucket, upload object', async () => {
testSmarts3Instance = await smarts3.Smarts3.createAndStart({
server: { port: TEST_PORT, silent: true, region: 'us-east-1' },
storage: { cleanSlate: true },
auth: {
enabled: true,
credentials: [{ accessKeyId: ACCESS_KEY, secretAccessKey: SECRET_KEY }],
},
});
authClient = new S3Client({
endpoint: BASE_URL,
region: 'us-east-1',
credentials: { accessKeyId: ACCESS_KEY, secretAccessKey: SECRET_KEY },
forcePathStyle: true,
});
await authClient.send(new CreateBucketCommand({ Bucket: BUCKET }));
await authClient.send(
new PutObjectCommand({
Bucket: BUCKET,
Key: 'test-obj.txt',
Body: 'hello policy eval',
ContentType: 'text/plain',
})
);
});
// ============================
// Principal matching
// ============================
tap.test('Principal: "*" → anonymous fetch GET succeeds', async () => {
await putPolicy([
{
Sid: 'PrincipalWildcard',
Effect: 'Allow',
Principal: '*',
Action: 's3:GetObject',
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
]);
const resp = await fetch(`${BASE_URL}/${BUCKET}/test-obj.txt`);
expect(resp.status).toEqual(200);
const text = await resp.text();
expect(text).toEqual('hello policy eval');
await clearPolicy();
});
tap.test('Principal: {"AWS": "*"} → anonymous GET fails, authenticated GET succeeds', async () => {
await putPolicy([
{
Sid: 'AwsWildcard',
Effect: 'Allow',
Principal: { AWS: '*' },
Action: 's3:GetObject',
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
]);
// Anonymous → no identity → Principal AWS:* doesn't match anonymous → NoOpinion → denied
const anonResp = await fetch(`${BASE_URL}/${BUCKET}/test-obj.txt`);
expect(anonResp.status).toEqual(403);
// Authenticated → has identity → Principal AWS:* matches → Allow
const authResp = await authClient.send(
new GetObjectCommand({ Bucket: BUCKET, Key: 'test-obj.txt' })
);
expect(authResp.$metadata.httpStatusCode).toEqual(200);
await clearPolicy();
});
tap.test('Principal: {"AWS": "arn:aws:iam::TESTAKID"} → authenticated GET succeeds', async () => {
await putPolicy([
{
Sid: 'SpecificPrincipal',
Effect: 'Allow',
Principal: { AWS: `arn:aws:iam::${ACCESS_KEY}` },
Action: 's3:GetObject',
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
]);
const resp = await authClient.send(
new GetObjectCommand({ Bucket: BUCKET, Key: 'test-obj.txt' })
);
expect(resp.$metadata.httpStatusCode).toEqual(200);
await clearPolicy();
});
tap.test('Principal: {"AWS": "arn:aws:iam::WRONGKEY"} → authenticated GET still succeeds (default allow)', async () => {
await putPolicy([
{
Sid: 'WrongPrincipal',
Effect: 'Allow',
Principal: { AWS: 'arn:aws:iam::WRONGKEY' },
Action: 's3:GetObject',
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
]);
// Principal doesn't match our key → NoOpinion → default allow for authenticated
const resp = await authClient.send(
new GetObjectCommand({ Bucket: BUCKET, Key: 'test-obj.txt' })
);
expect(resp.$metadata.httpStatusCode).toEqual(200);
await clearPolicy();
});
// ============================
// Action matching
// ============================
tap.test('Action: "s3:*" → anonymous can GET and PUT (wildcard matches all)', async () => {
await putPolicy([
{
Sid: 'S3Wildcard',
Effect: 'Allow',
Principal: '*',
Action: 's3:*',
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
]);
const getResp = await fetch(`${BASE_URL}/${BUCKET}/test-obj.txt`);
expect(getResp.status).toEqual(200);
const putResp = await fetch(`${BASE_URL}/${BUCKET}/anon-wildcard.txt`, {
method: 'PUT',
body: 'wildcard put',
});
expect(putResp.status).toEqual(200);
// Clean up the object we created
await authClient.send(new DeleteObjectCommand({ Bucket: BUCKET, Key: 'anon-wildcard.txt' }));
await clearPolicy();
});
tap.test('Action: "*" → global wildcard matches all actions', async () => {
await putPolicy([
{
Sid: 'GlobalWildcard',
Effect: 'Allow',
Principal: '*',
Action: '*',
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
]);
const getResp = await fetch(`${BASE_URL}/${BUCKET}/test-obj.txt`);
expect(getResp.status).toEqual(200);
const putResp = await fetch(`${BASE_URL}/${BUCKET}/anon-global.txt`, {
method: 'PUT',
body: 'global wildcard',
});
expect(putResp.status).toEqual(200);
await authClient.send(new DeleteObjectCommand({ Bucket: BUCKET, Key: 'anon-global.txt' }));
await clearPolicy();
});
tap.test('Action: "s3:Get*" → anonymous can GET but not PUT (prefix wildcard)', async () => {
await putPolicy([
{
Sid: 'PrefixWildcard',
Effect: 'Allow',
Principal: '*',
Action: 's3:Get*',
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
]);
const getResp = await fetch(`${BASE_URL}/${BUCKET}/test-obj.txt`);
expect(getResp.status).toEqual(200);
const putResp = await fetch(`${BASE_URL}/${BUCKET}/anon-prefix.txt`, {
method: 'PUT',
body: 'should fail',
});
expect(putResp.status).toEqual(403);
await clearPolicy();
});
tap.test('Action: ["s3:GetObject", "s3:PutObject"] → anonymous can GET and PUT but not DELETE', async () => {
await putPolicy([
{
Sid: 'MultiAction',
Effect: 'Allow',
Principal: '*',
Action: ['s3:GetObject', 's3:PutObject'],
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
]);
const getResp = await fetch(`${BASE_URL}/${BUCKET}/test-obj.txt`);
expect(getResp.status).toEqual(200);
const putResp = await fetch(`${BASE_URL}/${BUCKET}/anon-multi.txt`, {
method: 'PUT',
body: 'multi action',
});
expect(putResp.status).toEqual(200);
const delResp = await fetch(`${BASE_URL}/${BUCKET}/anon-multi.txt`, {
method: 'DELETE',
});
expect(delResp.status).toEqual(403);
// Clean up
await authClient.send(new DeleteObjectCommand({ Bucket: BUCKET, Key: 'anon-multi.txt' }));
await clearPolicy();
});
// ============================
// Resource ARN matching
// ============================
tap.test('Resource: "arn:aws:s3:::eval-bucket/*" → anonymous GET of object succeeds', async () => {
await putPolicy([
{
Sid: 'ResourceWildcard',
Effect: 'Allow',
Principal: '*',
Action: 's3:GetObject',
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
]);
const resp = await fetch(`${BASE_URL}/${BUCKET}/test-obj.txt`);
expect(resp.status).toEqual(200);
await clearPolicy();
});
tap.test('Resource: exact key → anonymous GET of that key succeeds, other key fails', async () => {
await putPolicy([
{
Sid: 'ExactResource',
Effect: 'Allow',
Principal: '*',
Action: 's3:GetObject',
Resource: `arn:aws:s3:::${BUCKET}/test-obj.txt`,
},
]);
const goodResp = await fetch(`${BASE_URL}/${BUCKET}/test-obj.txt`);
expect(goodResp.status).toEqual(200);
// Other key → resource doesn't match → NoOpinion → denied for anonymous
const badResp = await fetch(`${BASE_URL}/${BUCKET}/nonexistent.txt`);
expect(badResp.status).toEqual(403);
await clearPolicy();
});
tap.test('Resource: wrong bucket ARN → NoOpinion → anonymous GET denied', async () => {
await putPolicy([
{
Sid: 'WrongBucket',
Effect: 'Allow',
Principal: '*',
Action: 's3:GetObject',
Resource: 'arn:aws:s3:::other-bucket/*',
},
]);
const resp = await fetch(`${BASE_URL}/${BUCKET}/test-obj.txt`);
expect(resp.status).toEqual(403);
await clearPolicy();
});
tap.test('Resource: "*" → matches everything, anonymous GET succeeds', async () => {
await putPolicy([
{
Sid: 'StarResource',
Effect: 'Allow',
Principal: '*',
Action: 's3:GetObject',
Resource: '*',
},
]);
const resp = await fetch(`${BASE_URL}/${BUCKET}/test-obj.txt`);
expect(resp.status).toEqual(200);
await clearPolicy();
});
// ============================
// Deny-over-Allow priority
// ============================
tap.test('Allow + Deny same action → anonymous GET denied', async () => {
await putPolicy([
{
Sid: 'AllowGet',
Effect: 'Allow',
Principal: '*',
Action: 's3:GetObject',
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
{
Sid: 'DenyGet',
Effect: 'Deny',
Principal: '*',
Action: 's3:GetObject',
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
]);
const resp = await fetch(`${BASE_URL}/${BUCKET}/test-obj.txt`);
expect(resp.status).toEqual(403);
await clearPolicy();
});
tap.test('Allow s3:* + Deny s3:DeleteObject → anonymous GET succeeds, DELETE denied', async () => {
await putPolicy([
{
Sid: 'AllowAll',
Effect: 'Allow',
Principal: '*',
Action: 's3:*',
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
{
Sid: 'DenyDelete',
Effect: 'Deny',
Principal: '*',
Action: 's3:DeleteObject',
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
]);
const getResp = await fetch(`${BASE_URL}/${BUCKET}/test-obj.txt`);
expect(getResp.status).toEqual(200);
const delResp = await fetch(`${BASE_URL}/${BUCKET}/test-obj.txt`, { method: 'DELETE' });
expect(delResp.status).toEqual(403);
await clearPolicy();
});
tap.test('Statement order does not matter: Deny first, Allow second → still denied', async () => {
await putPolicy([
{
Sid: 'DenyFirst',
Effect: 'Deny',
Principal: '*',
Action: 's3:GetObject',
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
{
Sid: 'AllowSecond',
Effect: 'Allow',
Principal: '*',
Action: 's3:GetObject',
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
]);
const resp = await fetch(`${BASE_URL}/${BUCKET}/test-obj.txt`);
expect(resp.status).toEqual(403);
await clearPolicy();
});
// ============================
// NoOpinion fallback
// ============================
tap.test('NoOpinion: policy allows PutObject only → authenticated GET falls through (default allow)', async () => {
await putPolicy([
{
Sid: 'AllowPutOnly',
Effect: 'Allow',
Principal: '*',
Action: 's3:PutObject',
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
]);
// Authenticated → NoOpinion → default allow
const resp = await authClient.send(
new GetObjectCommand({ Bucket: BUCKET, Key: 'test-obj.txt' })
);
expect(resp.$metadata.httpStatusCode).toEqual(200);
await clearPolicy();
});
tap.test('NoOpinion: same policy → anonymous GET falls through → default deny (403)', async () => {
await putPolicy([
{
Sid: 'AllowPutOnly',
Effect: 'Allow',
Principal: '*',
Action: 's3:PutObject',
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
]);
// Anonymous → NoOpinion for GetObject → default deny
const resp = await fetch(`${BASE_URL}/${BUCKET}/test-obj.txt`);
expect(resp.status).toEqual(403);
await clearPolicy();
});
// ============================
// IAM action mapping
// ============================
tap.test('Policy allows s3:GetObject → anonymous HEAD object succeeds (HeadObject maps to s3:GetObject)', async () => {
await putPolicy([
{
Sid: 'AllowGet',
Effect: 'Allow',
Principal: '*',
Action: 's3:GetObject',
Resource: `arn:aws:s3:::${BUCKET}/*`,
},
]);
const resp = await fetch(`${BASE_URL}/${BUCKET}/test-obj.txt`, { method: 'HEAD' });
expect(resp.status).toEqual(200);
await clearPolicy();
});
tap.test('Policy allows s3:ListBucket → anonymous HEAD bucket succeeds', async () => {
await putPolicy([
{
Sid: 'AllowList',
Effect: 'Allow',
Principal: '*',
Action: 's3:ListBucket',
Resource: `arn:aws:s3:::${BUCKET}`,
},
]);
const resp = await fetch(`${BASE_URL}/${BUCKET}`, { method: 'HEAD' });
expect(resp.status).toEqual(200);
await clearPolicy();
});
tap.test('Policy allows s3:ListBucket → anonymous GET bucket (list objects) succeeds', async () => {
await putPolicy([
{
Sid: 'AllowList',
Effect: 'Allow',
Principal: '*',
Action: 's3:ListBucket',
Resource: `arn:aws:s3:::${BUCKET}`,
},
]);
const resp = await fetch(`${BASE_URL}/${BUCKET}`);
expect(resp.status).toEqual(200);
const text = await resp.text();
expect(text).toInclude('ListBucketResult');
await clearPolicy();
});
// ============================
// Teardown
// ============================
tap.test('teardown: clean up and stop server', async () => {
await authClient.send(new DeleteObjectCommand({ Bucket: BUCKET, Key: 'test-obj.txt' }));
await authClient.send(new DeleteBucketCommand({ Bucket: BUCKET }));
await testSmarts3Instance.stop();
});
export default tap.start();

View File

@@ -3,6 +3,6 @@
*/ */
export const commitinfo = { export const commitinfo = {
name: '@push.rocks/smarts3', name: '@push.rocks/smarts3',
version: '5.1.1', version: '5.3.0',
description: 'A Node.js TypeScript package to create a local S3 endpoint for simulating AWS S3 operations using mapped local directories for development and testing purposes.' description: 'A Node.js TypeScript package to create a local S3 endpoint for simulating AWS S3 operations using mapped local directories for development and testing purposes.'
} }

View File

@@ -58,6 +58,7 @@ export interface IServerConfig {
port?: number; port?: number;
address?: string; address?: string;
silent?: boolean; silent?: boolean;
region?: string;
} }
/** /**
@@ -89,6 +90,7 @@ const DEFAULT_CONFIG: ISmarts3Config = {
port: 3000, port: 3000,
address: '0.0.0.0', address: '0.0.0.0',
silent: false, silent: false,
region: 'us-east-1',
}, },
storage: { storage: {
directory: paths.bucketsDir, directory: paths.bucketsDir,