Chunking Jobs

Sometimes it's useful to split up the processing of something into multiple jobs, like processing a large file or scraping data from a paginated API. Laravel haystack uses the Laravel Chunkable Job package and allows you to split a job up into chunks.
Setup
Firstly, install the laravel-chunkable-jobs package using Composer with the command below.
composer require sammyjo20/laravel-chunkable-jobs
Configuring Jobs
Next, just create a job, remove the handle method and extend the ChunkableHaystackJob class. It's important that you extend this class and not ChunkableJob as it will use Haystack's methods to keep processing on the same Haystack.
You do not need to add the StackableJob interface or Stackable trait since the ChunkableHaystackJob will already add this for you.
<?php
​
use Sammyjo20\ChunkableJobs\Chunk;
use Sammyjo20\LaravelHaystack\ChunkableHaystackJob;
​
class GetPageOfPokemon extends ChunkableHaystackJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
​
    public function defineChunk(): ?Chunk
    {
        $response = Http::asJson()->get('https://pokeapi.co/api/v2/pokemon');
​
    	$count = $response->json('count'); // 1154
​
    	return new Chunk(totalItems: $count, chunkSize: 1, startingPosition: 1);
    }
​
    protected function handleChunk(Chunk $chunk): void
    {
        $response = Http::asJson()->get(sprintf('https://pokeapi.co/api/v2/pokemon?limit=%s&offset=%s', $chunk->limit, $chunk->offset));
​
    	$data = $response->json();
​
    	// Store data of response
    }
}
Documentation
To read more the laravel-chunkable-jobs documentation click here.
Last modified 11mo ago