File Streaming in PHP

Ok so here is the issue… I have figured out how to upload large files to my server and organize them using a chunking methods and database tables to track the files, directories and metadata. Curious to set that up, see this post Chunk Uploading Large Files From React to Laravel.

Now I wan to retrieve a byte stream for large video and data assets so I can port them in a folder to IPFS. In first attempts I ran into memory issues, then I ran into shitty, cryptic CORS issues with video/quicktime mimes.

Finally I found this methods to get any type of file of any size!! Or at least I have not hit the limit yet. I have test 2GB so far, crashed my CRA node instance a couple of times, but restarted and it worked.

Lets see how this works…

Magic!!

React Frontend

Here is the inside of a function in my react project. This function calls the

var bodyFormData = new FormData();
bodyFormData.append('_method', 'post');
bodyFormData.append('path', path);
axios.defaults.headers.post['Accept'] = 'application/json';
axios.defaults.withCredentials = true;
const response = await axios({
    method: "post",
    url: appContext.apiRoot + 'files/servestream',
    data: bodyFormData,
    headers: {
        //"Content-Type": "multipart/form-data",
        //"Access-Control-Allow-Origin": '*'
    },
    responseType: 'arraybuffer'
})
console.log(response);
/*  */
const buffer = Buffer.from(response.data, "utf-8")
object.path = path;
object.content = buffer;
files.push(object);

Lets break this down a bit…

• First setup Axios to communicate with Laravel properly, otherwise you will have nightmarish CORS issues.
• responseType: ‘arraybuffer’
• Then CREATE the buffer on the client side.

Laravel PHP Backend

$path = $request->path;
$storagePath = storage_path() . '/app/public/' . $path;
$mimeType = mime_content_type($storagePath);
$size = Storage::disk('public')->size($path);
//$fileName = $file->getClientOriginalName();       

$headers = array(
            'Content-Type' => 'application/octet-stream', //$mimeType, //application/octet-stream'
            'Content-Length' => $size,
            'Content-Disposition' => 'inline; filename="' . $storagePath . '"',
            'Content-Transfer-Encoding' => 'binary'
        );

        //if ($mimeType == 'video/quicktime') {
        $response = new StreamedResponse(
            function () use ($storagePath, $fileName) {
                // Open output stream
                if ($file = fopen($storagePath, 'rb')) {
                    while (!feof($file) and (connection_status() == 0)) {
                        print(fread($file, 1024 * 8));
                        flush();
                    }
                    fclose($file);
                }
            },
            200,
            $headers
        );

References and Resources

https://stackoverflow.com/questions/15942497/why-dont-large-files-download-easily-in-laravel

https://medium.com/@barryvdh/streaming-large-csv-files-with-laravel-chunked-queries-4158e484a5a2#.z9aey685l

• fwrite() – Binary-safe file write
• fopen() – Opens file or URL
• fsockopen() – Open Internet or Unix domain socket connection
• popen() – Opens process file pointer
• fgets() – Gets line from file pointer
• fgetss() – Gets line from file pointer and strip HTML tags
• fscanf() – Parses input from a file according to a format
• file() – Reads entire file into an array
• fpassthru() – Output all remaining data on a file pointer
• ftell() – Returns the current position of the file read/write pointer
• rewind() – Rewind the position of a file pointer
• unpack() – Unpack data from binary string