# Introduction

⚡️ Supercharged job chains for Laravel

{% hint style="warning" %}

#### Notice 14/05/2024

I am no longer going to be accepting new features for Laravel Haystack. I intend to still ensure security fixes are made, but I feel that the project is now complete. Additionally, I feel that Laravel's job batches and chains in Laravel 10+ are a lot more powerful and you may not need Laravel Haystack in 2024.
{% endhint %}

Laravel Haystack provides supercharged job chains for Laravel. It comes with powerful features like delaying jobs for as long as you like, applying middleware to every job, sharing data and models between jobs and even chunking jobs. Laravel Haystack supports every queue connection/worker out of the box. (Database, Redis/Horizon, SQS). It's great if you need to queue thousands of jobs in a chain or if you are looking for features that the original Bus chain doesn't provide.

```php
$haystack = Haystack::build()
   ->addJob(new RecordPodcast)
   ->addJob(new ProcessPodcast)
   ->addJob(new PublishPodcast)
   ->then(function () {
      // Haystack completed
   })
   ->catch(function () {
      // Haystack failed
   })
   ->finally(function () {
      // Always run either on success or fail.
   })
   ->withMiddleware([
      // Middleware for every job
   ])
   ->withDelay(60)
   ->withModel($user)
   ->dispatch();
```

#### But doesn't Laravel already have job chains?

Yep, Laravel does have job chains but there are quite limited and come with some disadvantages that you might want to think about.

* They consume quite a lot of memory/data since the chain is stored inside the job. This is especially true if you are storing thousands of jobs.
* They are volatile, meaning if you lose one job in the chain - you lose the whole chain.
* They do not provide the `then`, `catch`, `finally` callable methods that batched jobs do.
* Long delays with memory-based or SQS queue is not possible as you could lose the jobs due to expiry or if the server shuts down.
* You can't share data between jobs as there is no "state" across the chain

Laravel Haystack aims to solve this by storing the job chain in the database and queuing one job at a time. When the job is completed, Laravel Haystack listens out for the "job completed" event and queues the next job in the chain from the database.

#### Laravel Haystack Features

* Low memory consumption as one job is processed at a time and the chain is stored in the database
* You can delay/release jobs for as long as you want since it will use the scheduler to restart a chain. Even if your queue driver is SQS!
* It provides callback methods like `then`, `catch` and `finally`.
* Global middleware that can be applied to every single job in the chain
* You can store models and data that are shared with every job in the chain.
* You can prepare a Haystack and dispatch it at a later time

#### Use Cases

* If you need to make hundreds or thousands of API calls in a row, can be combined with Spatie's Job Rate Limiter to keep track of delays and pause jobs when a rate limit is hit.
* If you need to queue thousands of jobs in a chain at a time.
* If you need to batch import rows of data - each row can be a haystack job (bale) and processed one at a time. While keeping important job information stored in the database.
* If you need "release" times longer than 15 minutes if you are using Amazon SQS


# How It Works

Laravel Haystack works by storing your jobs on the database and then queuing the job onto the desired connection. The Flowchart below represents the flow Haystack follows.

<figure><img src="/files/K2cB40xoK2Zsj09V3tHg" alt=""><figcaption></figcaption></figure>


# Installation

You can install the package with Composer. **Laravel Haystack Requires Laravel 8+ and PHP 8.1**

```
composer require sammyjo20/laravel-haystack
```

Next, just run the installation command!

```
php artisan haystack:install
```


# Usage

Let's build our first Haystack. Start preparing your jobs by adding the **StackableJob** interface and **Stackable** trait. These will make them compatible with Haystack, however, you can still dispatch the jobs on their own with the trait and interface added.

```php
<?php
 
namespace App\Jobs;
 
use Sammyjo20\LaravelHaystack\Contracts\StackableJob;
use Sammyjo20\LaravelHaystack\Concerns\Stackable;
 
class ProcessPodcast implements ShouldQueue, StackableJob
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, Stackable
```

Now import the Haystack model and then call the `build` static function on it. This will provide you with an instance of the **HaystackBuilder** class which can be used to build your haystack. You can then use the `addJob` method to add jobs to the Haystack.

```php
<?php

use Sammyjo20\LaravelHaystack\Models\Haystack;

Haystack::build()
   ->addJob(new RecordPodcast)
   ->addJob(new PublishPodcast)
   ->addJob(new TweetAboutPodcast);
```

{% hint style="info" %}
You may also use the `addJobs` method which can accept an array or collection of jobs or the`addJobsWhen` method to conditionally add a job to the haystack.
{% endhint %}

#### Dispatching Haystacks

Finally, use the `dispatch` method to dispatch the haystack onto the queue connection that you have specified in your application's config. [See here for dispatching on custom connections](/next-up/connection-queue-and-delay).

<pre class="language-php"><code class="lang-php">&#x3C;?php

use Sammyjo20\LaravelHaystack\Models\Haystack;

$haystack = Haystack::build()
   ->addJob(new RecordPodcast)
   ->addJob(new PublishPodcast)
   ->addJob(new TweetAboutPodcast)
<strong>   ->dispatch();
</strong></code></pre>

#### Creating Haystacks For Later

Sometimes you may wish to perform other logic or wait before dispatching the Haystack. To accomplish this, use the `create` method instead which will create the Haystack model in the database but will not dispatch it. When you are ready to dispatch it, retrieve the model and invoke the `start` method.

<pre class="language-php"><code class="lang-php">&#x3C;?php

use Sammyjo20\LaravelHaystack\Models\Haystack;

$haystack = Haystack::build()
   ->addJob(new RecordPodcast)
   ->addJob(new PublishPodcast)
   ->addJob(new TweetAboutPodcast)
   ->create();

// Do other things...

<strong>$haystack->start(); // Initiate haystack
</strong></code></pre>

#### Cancelling Haystacks

If you need to cancel the Haystack during processing, you can do this by using the `cancel` method on the Haystack model. If a job is being processed when you cancel it, it will process the next job and then stop before the job is executed.

<pre class="language-php"><code class="lang-php">use Sammyjo20\LaravelHaystack\Models\Haystack;

$haystack = Haystack::build()
   ->addJob(new RecordPodcast)
   ->addJob(new PublishPodcast)
   ->addJob(new TweetAboutPodcast)
   ->create();

// Store Haystack

$haystack->start();

// Do other things...

<strong>$haystack->cancel();
</strong></code></pre>

{% hint style="info" %}
When a haystack is cancelled, we will not run the `then` or `catch` closures, but we will execute the `finally` closure.
{% endhint %}


# Configuration

Laravel Haystack provides various configuration options, altering default package behaviour that you may find useful for your application. These can be changed in the `config/haystack.php` file.

## Return All Haystack Data When Finished

When this option is set to `true`, Laravel Haystack will query all the haystack data rows from the database and include them in the `then`/`finally`/`catch` callbacks as a Laravel collection upon completion of the job processing:

```php
// config/haystack.php

'return_all_haystack_data_when_finished' => true,
```

```php
use Illuminate\Support\Collection;

$haystack = Haystack::build()
   ->addJob(new RecordPodcast)
   ->then(function (Collection $data) {
       // ...
   })
   ->catch(function (Collection $data) {
       // ...
   })
   ->finally(function (Collection $data) {
       // ...
   })
   ->dispatch();
```

## Process Automatically

This configuration determines whether Laravel Haystack should automatically queue `Stackable` jobs after each job is processed. If set to `false`, you will need to manually call `$this->nextJob` inside your jobs:

```php
// config/haystack.php

'process_automatically' => false,
```

```php
<?php
 
namespace App\Jobs;
 
use Sammyjo20\LaravelHaystack\Contracts\StackableJob;
use Sammyjo20\LaravelHaystack\Concerns\Stackable;
 
class ProcessPodcast implements ShouldQueue, StackableJob
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, Stackable;
    
    public function handle()
    {
        // ...
        
        // Call the next job in the haystack:
        $this->nextJob();
        
        // Or, with a delay in seconds:
        $this->nextJob($seconds = 15);
        
        // Or, with a delay using a Carbon instance:
        $this->nextJob(now()->addDay());
    }
}
```

## Keep Stale Haystacks for Days

Defines the duration (in days) for which "stale" haystacks are retained. Stale haystacks are those where the controlling job has failed without sending the failure signal to laravel-haystack.

```php
// config/haystack.php

'keep_stale_haystacks_for_days' => 3,
```

## Delete Finished Haystacks

Determines whether Laravel Haystack should automatically delete haystacks after they have finished processing. If set to `false`, ensure to use the scheduled command to clean up old finished haystacks.

```php
// config/haystack.php

'delete_finished_haystacks' => true,
```

## Keep Finished Haystacks for Days

Specifies the duration (in days) for which finished haystacks will be retained. This is only applicable if `delete_finished_haystacks` is set to `false`.

```php
// config/haystack.php

'keep_finished_haystacks_for_days' => 1,
```

## Database Connection

Specifies the database connection used to store haystack jobs. The default value is retrieved from the `HAYSTACK_DB_CONNECTION` environment variable, falling back to the default database connection specified in your Laravel configuration (`DB_CONNECTION`).

```php
// config/haystack.php

'db_connection' => env(
    'HAYSTACK_DB_CONNECTION',
    env('DB_CONNECTION', 'mysql')
),
```


# Callback Events

There are four callback events that Haystack provides throughout its lifecycle. These are **then, catch, finally** and **paused**. You can use these methods to run application code like notifying a user.

{% hint style="warning" %}
Since these callback events are serialized and stored in the database you cannot use **$this** inside of the anonymous functions.
{% endhint %}

#### Then

The “then” event is triggered when the haystack has been completed successfully.

```php
$haystack = Haystack::build()
   ->addJob(new RecordPodcast)
   ->then(function () {
       // Do something... 
   })
   ->dispatch();
```

#### Catch

The “catch” event is triggered when the haystack has failed. You will still have the failed job to see what the error was, but this is useful if you need to perform any cleanup.

```php
$haystack = Haystack::build()
   ->addJob(new RecordPodcast)
   ->catch(function () {
       // Do something... 
   })
   ->dispatch();
```

#### Finally

The “finally “event is always triggered at the end of a haystack on both success and failure.&#x20;

```php
$haystack = Haystack::build()
   ->addJob(new RecordPodcast)
   ->finally(function () {
       // Do something... 
   })
   ->dispatch();
```

#### Paused

The "paused" event is triggered if the Haystack has been paused using the `pauseHaystack` method or a job has been released using the `longRelease` method. This is useful if you need to update the database to mark an import as paused, especially if the pause is for a long time.

```php
$haystack = Haystack::build()
   ->addJob(new RecordPodcast)
   ->paused(function () {
       // Do something... 
   })
   ->dispatch();
```

#### Invokable classes

Each of these methods supports invokable classes. If you use invokable classes you will have access to the **$this** context.&#x20;

```php
$haystack = Haystack::build()
   ->addJob(new RecordPodcast)
   ->then(new Then)
   ->catch(new Catch)
   ->finally(new Finally)
   ->paused(new Paused)
   ->dispatch();

// Example Invokable class

class Then {
   public function __invoke()
   {
       // Do something...
   }
}
```

#### Chained Methods

Each of the callback events can be chained for multiple events.

```php
$haystack = Haystack::build()
   ->addJob(new RecordPodcast)
   ->then(function () {
       // Do something first...
   })
   ->then(function () {
       // Then do something after! 
   })
   ->dispatch();
```


# Shared Data

Laravel Haystack has the ability for your jobs to store and retrieve state/data between jobs. This is really useful if you need to store data in the first job and then in the second job, process the data and in the final job, email the processed data. You are able to create a process pipeline since each job is processed in sequential order. This is really exciting because with traditional chained jobs, you cannot share data between jobs.

```php
<?php

$haystack = Haystack::build()
   ->addJob(new RetrieveDataFromApi)
   ->addJob(new ProcessDataFromApi)
   ->addJob(new StoreDataFromApi)
   ->dispatch();
```

#### Storing data inside of jobs

Inside your job, you can use the `setHaystackData()` method to store some data. This method accepts a key, value and optional [Eloquent cast](https://laravel.com/docs/eloquent-mutators).

```php
<?php
 
namespace App\Jobs;
 
use Sammyjo20\LaravelHaystack\Contracts\StackableJob;
use Sammyjo20\LaravelHaystack\Concerns\Stackable;
 
class RetrieveDataFromApi implements ShouldQueue, StackableJob
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, Stackable

    public function handle()
    {
        // Your application code...
        
        $this->setHaystackData('username', 'Sammyjo20');
    }
```

#### Casting data

The `setHaystackData` method supports any data type. It supports fully casting your data into any of [Eloquent's existing casts](https://laravel.com/docs/eloquent-mutators), or even your custom casts. Just provide a third argument to specify the cast.

```php
<?php
 
namespace App\Jobs;
 
use Sammyjo20\LaravelHaystack\Contracts\StackableJob;
use Sammyjo20\LaravelHaystack\Concerns\Stackable;
 
class RetrieveDataFromApi implements ShouldQueue, StackableJob
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, Stackable

    public function handle()
    {
        // Your application code...
        
        // Array Data, provide third argument to specify cast.
        
        $this->setHaystackData('data', ['username' => 'Sammyjo20'], 'array');
        
        // Carbon dates...
        
        $this->setHaystackData('currentDate', now(), 'immutable_datetime');
        
        // Supports custom casts
        
        $this->setHaystackData('customData', $object, CustomCast::class);
    }
```

#### Retrieving data inside of jobs

From one job you can set the data, but that data will be available to every job in the haystack there after. Just use the `getHaystackData` method to get data by key or use the `allHaystackData` to get a collection containing the haystack data.

```php
<?php
 
namespace App\Jobs;
 
use Sammyjo20\LaravelHaystack\Contracts\StackableJob;
use Sammyjo20\LaravelHaystack\Concerns\Stackable;
 
class RetrieveDataFromApi implements ShouldQueue, StackableJob
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, Stackable

    public function handle()
    {
        // Get data by key
        
        $username = $this->getHaystackData('username'); // Sammyjo20
        
        // Get all data
        
        $allData = $this->allHaystackData(); // Collection: ['username' => 'Sammyjo20']
    }
```

#### Getting The Data After Processing

Laravel Haystack will conveniently pass a collection into your then/catch/finally closures with all the of the data that you stored inside the Haystack. You can then use this data however you wish.

```php
<?php

$haystack = Haystack::build()
   ->addJob(new RetrieveDataFromApi)
   ->addJob(new ProcessDataFromApi)
   ->addJob(new StoreDataFromApi)
   ->then(function ($data) {
        // $data: ['username' => 'Sammyjo20', 'other-key' => 'value', 'collection' => new Collection],
   })
   ->catch(function ($data) {
        // $data: ['username' => 'Sammyjo20', 'other-key' => 'value', 'collection' => new Collection],
   })
   ->finally(function ($data) {
        // $data: ['username' => 'Sammyjo20', 'other-key' => 'value', 'collection' => new Collection],
   })
   ->dispatch();
```

If you would like to disable this functionality, you can provide the `dontReturnData` method to the Haystack builder. If this method is provided, Haystack won't run the query that retrieves all the data at the end of a Haystack.

```php
<?php

$haystack = Haystack::build()
   ->addJob(new RetrieveDataFromApi)
   ->addJob(new ProcessDataFromApi)
   ->addJob(new StoreDataFromApi)
   ->dontReturnData()
   ->then(function ($data) {
        // $data: null
   })
   ->dispatch();
```

If you would like to disable this feature entirely, you can set the `return_all_haystack_data_when_finished` config variable to false.

### Setting data when creating haystacks

Sometimes it's useful to set some data for your jobs to consume without having to pass every piece of data down into each job. You can use the `withData` method while you are building your Haystack to add data before the Haystack starts. It also accepts a key, value and optional Eloquent cast.

```php
<?php

$haystack = Haystack::build()
   ->addJob(new RetrieveDataFromApi)
   ->addJob(new ProcessDataFromApi)
   ->addJob(new StoreDataFromApi)
   ->withData('username', 'Sammyjo20')
   ->dispatch();
```


# Shared Models

You can also provide models when creating Haystacks which will be accessible to every job. This is extremely useful as you don't have to pass the model into every job. Just use the `withModel` method when building your Haystack to store a model.&#x20;

<pre class="language-php"><code class="lang-php">&#x3C;?php

$user = Auth::user();

Haystack::build()
    ->addJob(new RecordPodcast)
<strong>    ->withModel($user)
</strong>    ->dispatch();
</code></pre>

You can also provide an optional `$key` as the second argument.&#x20;

<pre class="language-php"><code class="lang-php">&#x3C;?php

$user = Auth::user();

Haystack::build()
    ->addJob(new RecordPodcast)
<strong>    ->withModel($user, 'admin')
</strong>    ->dispatch();
</code></pre>

Then, inside your jobs you can call the `getHaystackModel` method. If you did not provide a key, you can just pass in the model's class name.

```php
$user = $this->getHaystackModel(User::class);

// Or if you specified a key...

$this->getHaystackModel('admin');
```

{% hint style="info" %}
Laravel Haystack will serialize your models including their loaded relationships. It will only retrieve the model and the relationships when you attempt to get the model.
{% endhint %}


# Long Delays & Pauses

Haystack fully supports the existing `release` and `delay` methods in Laravel, however occasionally you may want to pause the haystack for an extended period of time, or release a job until the next day when an API rate limit is lifted. This can also be used as a longer delay if you are using Amazon SQS which only has a delay of 15 minutes.

Laravel Haystack can provide this by storing the resume date in the database and using the Scheduler to dispatch the haystack when it is ready. When Laravel Haystack is paused, no job is left in your queue.

#### Setting Up

Before you configure long releases or pauses, you must make sure your scheduler is running and the following line is added to your Console/Kernel.php file. If you can, provide the `onOneServer` method as well which will prevent any accidental duplicate resumes.

```php
<?php

// Add to Console/Kernel.php

$schedule->command('haystacks:resume')->everyMinute();

// One server if you are running the scheduler on multiple servers

$schedule->command('haystacks:resume')->everyMinute()->onOneServer();
```

#### Long Releasing

If you would like to release the current job back onto the queue, just use the `longRelease` method inside your job’s handle method. You can provide an integer for seconds or a Carbon datetime instance.

```php
<?php
 
namespace App\Jobs;
 
use Sammyjo20\LaravelHaystack\Contracts\StackableJob;
use Sammyjo20\LaravelHaystack\Concerns\Stackable;
 
class ProcessPodcast implements ShouldQueue, StackableJob
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, Stackable

    public function handle()
    {
        // Your application code...

        $this->longRelease(300); // Release for 5 minutes

        // Or use Carbon

        $this->longRelease(now()->addDays(2)); // Release for 2 days.
    }
```

#### Pausing the next job

If you want to process the current job but pause the Haystack for the next job, use the `pauseHaystack` method. If you have disabled automatic processing, you can provide a delay to the `nextJob` method.

```php
<?php
 
namespace App\Jobs;
 
use Sammyjo20\LaravelHaystack\Contracts\StackableJob;
use Sammyjo20\LaravelHaystack\Concerns\Stackable;
 
class ProcessPodcast implements ShouldQueue, StackableJo
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, Stackable

    public function handle()
    {
        // Your application code...

        $this->pauseHaystack(now()->addHours(4)); // Pause the haystack for 4 hours.
    }
```


# Appending & Prepending Jobs

You can append to the haystack inside a job. The appended job will go at the end of the chain. Just use the `appendToHaystack` method. If you would like to append a job to the haystack to be processed immediately, use the `prependToHaystack` method.

```php
<?php
 
namespace App\Jobs;
 
use Sammyjo20\LaravelHaystack\Contracts\StackableJob;
use Sammyjo20\LaravelHaystack\Concerns\Stackable;
 
class ProcessPodcast implements ShouldQueue, StackableJob
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, Stackable

    public function handle()
    {
        // Append a job to the end of the haystack

        $this->appendToHaystack(new DifferentJob);
        
        // Append a job and put it right at the top of the haystack
        
        $this->prependToHaystack(new NextJob);
    }
```

> The `appendToHaystack` and `prependToHaystack` methods also accept an array or Collection of jobs.


# Chunking Jobs

Sometimes it's useful to split up the processing of something into multiple jobs, like processing a large file or scraping data from a paginated API. Laravel haystack uses the Laravel Chunkable Job package and allows you to split a job up into chunks.

### Setup

Firstly, install the `laravel-chunkable-jobs` package using Composer with the command below.

```
composer require sammyjo20/laravel-chunkable-jobs
```

### Configuring Jobs

Next, just create a job, remove the `handle` method and extend the `ChunkableHaystackJob` class. It's important that you extend this class and not `ChunkableJob` as it will use Haystack's methods to keep processing on the same Haystack.

You do not need to add the `StackableJob` interface or `Stackable` trait since the `ChunkableHaystackJob` will already add this for you.

```php
<?php

use Sammyjo20\ChunkableJobs\Chunk;
use Sammyjo20\LaravelHaystack\ChunkableHaystackJob;

class GetPageOfPokemon extends ChunkableHaystackJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public function defineChunk(): ?Chunk
    {
        $response = Http::asJson()->get('https://pokeapi.co/api/v2/pokemon');

    	$count = $response->json('count'); // 1154

    	return new Chunk(totalItems: $count, chunkSize: 1, startingPosition: 1);
    }

    protected function handleChunk(Chunk $chunk): void
    {
        $response = Http::asJson()->get(sprintf('https://pokeapi.co/api/v2/pokemon?limit=%s&offset=%s', $chunk->limit, $chunk->offset));

    	$data = $response->json();

    	// Store data of response
    }
}
```

### Documentation

To read more the laravel-chunkable-jobs documentation [click here](https://github.com/sammyjo20/laravel-chunkable-jobs).


# Allowing Failed Jobs

The default behaviour of Haystack is to cancel the Haystack if one of the jobs in the chain fails. If you would like to continue processing the rest of the Haystack even if there was a failure, use the `allowFailures` option when building a Haystack.

```php
<?php

$haystack = Haystack::build()
   ->addJob(new RecordPodcast) 
   ->addJob(new ProcessPodcast)
   ->allowFailures()
   ->dispatch();
```

{% hint style="info" %}
When jobs fail they will be added to your `failed_jobs` database table.
{% endhint %}


# Global Middleware

You can also provide middleware that will be applied to every job in the haystack. It will accept either an array, a closure that returns an array or an invokable class that returns an array.&#x20;

This is useful if you don’t want to manually add the middleware to every job, or if the middleware cannot belong to the job on its own. One example of this being useful is if you want to apply an API rate limiter to your jobs that are making requests to a third party API.&#x20;

To add middleware to every job, use the `addMiddleware` method when building the haystack.

If you are unfamiliar with job middleware, [click here](https://laravel.com/docs/queues#job-middleware).

#### Array

```php
$haystack = Haystack::build()
   ->onQueue('podcasts')
   ->addJob(new RecordPodcast) 
   ->addJob(new ProcessPodcast)
   ->addMiddleware([
       (new RateLimited)->allows(30)->everyMinute(),
       new OtherMiddleware,
   ])
   ->dispatch();
```

#### Closure

You must return an array in the closure for it to work.

```php
$haystack = Haystack::build()
   ->onQueue('podcasts')
   ->addJob(new RecordPodcast) 
   ->addJob(new ProcessPodcast)
   ->addMiddleware(function () {
        return [
           (new RateLimited)->allows(30)->everyMinute(),
           new OtherMiddleware,
        ];
   })
   ->dispatch();
```

#### Invokable class

You must return an array inside your invokable class for it to work.

```php
$haystack = Haystack::build()
   ->onQueue('podcasts')
   ->addJob(new RecordPodcast) 
   ->addJob(new ProcessPodcast)
   ->addMiddleware(new PodcastMiddleware)
   ->dispatch();

// Invokable class...

class PodcastMiddleware {
    public function __invoke()
    {
       return [
          (new RateLimited)->allows(30)->everyMinute(),
           new OtherMiddleware,
       ];
    }
}
```

#### Chainable Middleware

You can also chain multiple middlewares together.

```php
$haystack = Haystack::build()
   ->onQueue('podcasts')
   ->addJob(new RecordPodcast) 
   ->addJob(new ProcessPodcast)
   ->addMiddleware([
       (new RateLimited)->allows(30)->everyMinute(),
       new OtherMiddleware,
   ])
   ->addMiddleware([
       new AnotherMiddleware,
   ])
   ->dispatch();
```


# Connection, Queue & Delay

You can configure a global delay, connection and queue that will apply to all jobs in the haystack. You can also provide a per-job configuration if you would prefer.

#### Delay

You can use the `withDelay` method to apply a global delay to every job.

```php
$haystack = Haystack::build()
   ->withDelay(60)
   ->addJob(new RecordPodcast) 
   ->addJob(new ProcessPodcast)
   ->dispatch();
```

#### Connection

You can use the `onConnection` method to use a given connection for every job.

```php
$haystack = Haystack::build()
   ->onConnection('redis')
   ->addJob(new RecordPodcast) 
   ->addJob(new ProcessPodcast)
   ->dispatch();
```

#### Queue

You can use the `onQueue` method to use a given queue for every job.

```php
$haystack = Haystack::build()
   ->onQueue('podcasts')
   ->addJob(new RecordPodcast) 
   ->addJob(new ProcessPodcast)
   ->dispatch();
```

#### Custom Delay, Connection, Queue Per Job

You can also choose to use a different delay, connection or queue for every job!

```php
$haystack = Haystack::build()
   ->onQueue('podcasts')
   ->addJob(new RecordPodcast, delay: 60, queue: 'low', connection: 'redis') 
   ->addJob(new ProcessPodcast, delay: 120, queue: 'high', connection: 'sqs')
   ->dispatch();
```

> If you have already configured the job with delay, connection or queue, it will use that configuration.


# Naming Haystacks

You may wish to give your haystack a custom name. This is especially useful for debugging, as you could check your database and see what haystacks are currently being processed. To give the haystack a name, use the `withName` method when building the haystack.

```php
<?php

$haystack = Haystack::build()
   ->withName('Process API Data')
   ->addJob(new RetrieveDataFromApi)
   ->addJob(new ProcessDataFromApi)
   ->addJob(new StoreDataFromApi)
```


# Before Saving Hook

Sometimes you may wish to modify the Haystack model before it is saved. You can use the `beforeSave` method to modify the Haystack model instance.

```php
<?php

Haystack::build()
    ->addJob(new RecordPodcast)
    ->beforeSave(function (Haystack $haystack) {
         $haystack->options->customOption = true;
     })
    ->dispatch();
```


# Custom Options

Every Haystack has a `HaystackOptions` class stored against it. This class is serialized and stored in the database and contains various configuration variables used when processing jobs. You may wish to specify a new custom option by using the `setOption` method when building Haystacks.

```php
<?php

Haystack::build()
    ->addJob(new RecordPodcast)
    ->setOption('someCustomOption', 'option-value')
    ->dispatch();
```

{% hint style="info" %}
Since the options are serialized into one column, it's recommended that you only store small amounts of data inside this class, and make sure what you are storing can be serialized.
{% endhint %}

Inside your jobs, you can use the `getHaystackOptions` or `getHaystackOption` method to retrieve the options.

```php
<?php

$this->getHaystackOptions(); // HaystackOptions.php

$this->getHaystackOption('someCustomOption'); // "option-value"
```


# Deleting Stale Haystacks

Laravel Haystack will attempt to clean up every job on successful/failed haystacks, however there may be a situation, especially if you have not enabled the automatic processing. If you have disabled the option to automatically delete haystacks when they are finished they may build up quickly.

You can prevent this by running the following prune command every day in your scheduler:

```php
<?php

// Add to Console/Kernel.php

use Sammyjo20\LaravelHaystack\Models\Haystack;

$schedule->command('model:prune', [
    '--model' => [Haystack::class],
])->daily();
```


# Deleting Specific Haystacks

If a haystack was created by mistake, and you would like to delete it without directly accessing the database, you may use the `haystacks:forget` command. The `haystacks:forget` command accepts the ID of the haystack as its only argument:

```
php artisan haystacks:forget <id>
```


# Clearing All Haystacks

If you would like to clear all haystacks from the database, you may use the `haystacks:clear` command. The `haystacks:clear` command accepts no arguments:

```
php artisan haystacks:clear
```


# Support

While I never expect anything, if you would like to support my work, you can donate to my Ko-Fi page by simply buying me a coffee or two!

<https://ko-fi.com/sammyjo20>