databases storage and caching

Object Storage Orientation: S3-Compatible Storage

The main skill is understanding the boundary. PHP usually stores metadata in the database and stores the file bytes in object storage. The application then uses a bucket, object key, content type, access policy, and sometimes a signed URL to manage the file safely.

Buckets, Objects, And Keys

A bucket is a container. An object is one stored file-like value. A key is the object's path-like identifier inside the bucket.

Object storage keys look like paths, but they are not normal directories. They are strings used to identify objects.

PHP example
<?php

declare(strict_types=1);

function invoiceObjectKey(int $accountId, int $invoiceId): string
{
    return sprintf('accounts/%d/invoices/%d.pdf', $accountId, $invoiceId);
}

echo invoiceObjectKey(17, 9001) . PHP_EOL;

// Prints:
// accounts/17/invoices/9001.pdf

Good keys are predictable, avoid raw user filenames, and include enough information to organise the object. Do not trust a browser-uploaded filename as the storage key.

Store Metadata In The Database

The database should usually store the relationship between an application record and the object key. The object storage service stores the bytes.

PHP example
<?php

declare(strict_types=1);

function uploadedDocumentRecord(int $userId, string $objectKey, string $originalName): array
{
    return [
        'user_id' => $userId,
        'object_key' => $objectKey,
        'original_name' => $originalName,
        'visibility' => 'private',
    ];
}

print_r(uploadedDocumentRecord(
    42,
    'users/42/documents/8f4b-report.pdf',
    'report.pdf'
));

// Prints:
// [user_id] => 42
// [object_key] => users/42/documents/8f4b-report.pdf
// [visibility] => private

This keeps application queries fast and simple. For example, the database can answer "which documents belong to this user?" without listing the whole bucket.

Public And Private Objects

Most application uploads should start private. A private object cannot be downloaded just because someone knows the key. The application checks authorisation first, then either streams the file through PHP or creates a short-lived signed URL.

Public objects are suitable for assets that anyone may read, such as public product images or downloadable brochures. Do not make a whole bucket public unless every object in it is safe to expose.

PHP example
<?php

declare(strict_types=1);

function accessPlan(string $visibility, bool $userIsAllowed): string
{
    if ($visibility === 'public') {
        return 'serve using the public object URL';
    }

    if ($userIsAllowed) {
        return 'create a short-lived signed URL';
    }

    return 'deny access';
}

echo accessPlan('private', true) . PHP_EOL;
echo accessPlan('private', false) . PHP_EOL;

// Prints:
// create a short-lived signed URL
// deny access

Signed URLs are useful because the storage service serves the file directly, but only for a limited time and only for that object.

Validate Uploads Before Storing

User uploads need validation before they reach object storage. Check size, MIME type, extension policy, and the business rule for who may upload.

PHP example
<?php

declare(strict_types=1);

function validateUpload(string $originalName, string $mimeType, int $bytes): array
{
    $allowedMimeTypes = ['application/pdf', 'image/jpeg', 'image/png'];
    $maxBytes = 5 * 1024 * 1024;

    if ($bytes <= 0 || $bytes > $maxBytes) {
        return ['valid' => false, 'reason' => 'file size is not allowed'];
    }

    if (!in_array($mimeType, $allowedMimeTypes, true)) {
        return ['valid' => false, 'reason' => 'file type is not allowed'];
    }

    return ['valid' => true, 'reason' => 'upload can be stored'];
}

print_r(validateUpload('report.pdf', 'application/pdf', 120_000));

// Prints:
// [valid] => 1
// [reason] => upload can be stored

The original filename can be stored for display, but it should not decide the storage key by itself.

Generate Storage Keys Safely

A good upload key is controlled by the application. It may include the owner ID, a domain folder, a random token, and a cleaned extension.

PHP example
<?php

declare(strict_types=1);

function extensionFromMimeType(string $mimeType): string
{
    return match ($mimeType) {
        'application/pdf' => 'pdf',
        'image/jpeg' => 'jpg',
        'image/png' => 'png',
        default => 'bin',
    };
}

function uploadObjectKey(int $userId, string $mimeType, string $randomHex): string
{
    return sprintf(
        'users/%d/uploads/%s.%s',
        $userId,
        $randomHex,
        extensionFromMimeType($mimeType)
    );
}

echo uploadObjectKey(42, 'application/pdf', 'a1b2c3d4') . PHP_EOL;

// Prints:
// users/42/uploads/a1b2c3d4.pdf

In real code, generate the random part with something like bin2hex(random_bytes(16)). The example accepts the random value as an argument so the output is stable.

Content Type And Download Behaviour

When uploading an object, set metadata such as Content-Type. Without it, browsers may download or display files incorrectly. Some applications also set Content-Disposition so a file downloads with a friendly filename.

PHP example
<?php

declare(strict_types=1);

function objectUploadOptions(string $mimeType, string $downloadName): array
{
    return [
        'ContentType' => $mimeType,
        'ContentDisposition' => 'attachment; filename="' . addslashes($downloadName) . '"',
        'Visibility' => 'private',
    ];
}

print_r(objectUploadOptions('application/pdf', 'report.pdf'));

// Prints:
// [ContentType] => application/pdf
// [Visibility] => private

The exact option names vary by library. The concept is stable: upload bytes plus metadata, then store the object key in the application database.

Failure And Cleanup

Uploading a file and inserting a database row is not one automatic transaction across two systems. One can succeed while the other fails.

Common approaches include:

  • upload the object, then insert the database row, and delete the object if the row insert fails;
  • insert a pending database row, upload the object, then mark the row complete;
  • run a cleanup job that removes orphaned objects or stale pending rows;
  • make repeated upload attempts idempotent by using a stable object key for the same logical file.

This matters in job work because storage bugs often show up as missing files, orphaned files, duplicate uploads, or records pointing at objects that no longer exist.

Large Files And Direct Uploads

Small files can be uploaded through PHP. Large files are often uploaded directly from the browser to object storage using a signed upload URL. That avoids pushing large request bodies through the application server.

For very large files, object storage services often support multipart uploads. The details depend on the provider and SDK, but the design question is the same: where is upload state tracked, how are failed uploads cleaned up, and when is the file considered complete?

What To Check

Before moving on, make sure you can:

  • explain that object storage stores file bytes while the database stores references and metadata;
  • describe buckets, object keys, content types, visibility, and signed URLs;
  • avoid trusting user filenames as storage keys;
  • validate file size and type before storing uploads;
  • keep private files private until authorisation passes;
  • plan cleanup for failed uploads and orphaned objects;
  • choose object storage for file-like data instead of putting large file bytes in database rows.

Practice

Practice: Design An Object Storage Upload Plan

Build a small PHP helper that plans how a user-uploaded document should be stored in S3-compatible object storage.

Requirements

  • Validate that the file size is positive and no larger than 5 MB.
  • Allow only PDF, JPEG, and PNG MIME types.
  • Generate an application-controlled object key using the user ID, a random token, and the extension derived from the MIME type.
  • Return upload options that include content type and private visibility.
  • Return a database record shape that stores the object key and original filename.
  • Show one accepted upload and one rejected upload.

You do not need to connect to a real S3 service. Model the decision and the data that the application would pass to a storage client.

Show solution

This solution keeps the object key under application control and stores only the reference data that the database needs.

PHP example
<?php

declare(strict_types=1);

function uploadExtension(string $mimeType): ?string
{
    return match ($mimeType) {
        'application/pdf' => 'pdf',
        'image/jpeg' => 'jpg',
        'image/png' => 'png',
        default => null,
    };
}

function planObjectUpload(
    int $userId,
    string $originalName,
    string $mimeType,
    int $bytes,
    string $randomHex
): array {
    $maxBytes = 5 * 1024 * 1024;
    $extension = uploadExtension($mimeType);

    if ($bytes <= 0 || $bytes > $maxBytes) {
        return [
            'accepted' => false,
            'reason' => 'file size is not allowed',
        ];
    }

    if ($extension === null) {
        return [
            'accepted' => false,
            'reason' => 'file type is not allowed',
        ];
    }

    $objectKey = sprintf('users/%d/uploads/%s.%s', $userId, $randomHex, $extension);

    return [
        'accepted' => true,
        'object_key' => $objectKey,
        'upload_options' => [
            'ContentType' => $mimeType,
            'Visibility' => 'private',
        ],
        'database_record' => [
            'user_id' => $userId,
            'object_key' => $objectKey,
            'original_name' => $originalName,
        ],
    ];
}

$accepted = planObjectUpload(42, 'report.pdf', 'application/pdf', 120_000, 'a1b2c3d4');
$rejected = planObjectUpload(42, 'script.php', 'application/x-php', 2_000, 'ffffeeee');

echo $accepted['object_key'] . PHP_EOL;
echo $accepted['upload_options']['Visibility'] . PHP_EOL;
echo $rejected['reason'] . PHP_EOL;

// Prints:
// users/42/uploads/a1b2c3d4.pdf
// private
// file type is not allowed

The plan keeps the uploaded filename as display metadata, not as the storage key. The application can now upload the bytes to object storage, insert the database record, and clean up the object if the database write fails.