Sidekiq Encrypted Args

Support for encrypting arguments for Sidekiq.

Problem

Sidekiq stores the arguments for jobs as JSON in Redis. If your workers include sensitive information (API keys, passwords, personally identifiable information, etc.), you run the risk of accidentally exposing this information. Job arguments are visible in the Sidekiq web interface and your security will only be as good as your Redis server security.

This can be an even bigger issue if you use scheduled jobs since sensitive data on those jobs will live in Redis until the job is run. Data written to Redis can also be persisted to disk and live on long after the data in Redis has been deleted.

Solution

This gem adds Sidekiq middleware that allows you to specify job arguments for your workers that should be encrypted in Redis. You do this by adding encrypted_args to the sidekiq_options in the worker. Jobs for these workers will have their arguments encrypted in Redis and decrypted when passed to the perform method.

To use the gem, you will need to specify a secret that will be used to encrypt the arguments as well as add the middleware to your Sidekiq client and server middleware stacks. You can set that up by adding this to the end of your Sidekiq initialization:

Sidekiq::EncryptedArgs.configure!(secret: "YourSecretKey")

If the secret is not set explicitly, the value of the SIDEKIQ_ENCRYPTED_ARGS_SECRET environment variable will be used as the secret. If you call configure! without setting a secret (either explicitly or via the environment variable), an error will be raised.

The call to Sidekiq::EncryptedArgs.configure! will prepend the client encryption middleware and append server decryption middleware. By doing this, any other middleware you register will only receive the encrypted parameters (e.g. logging middleware will receive the encrypted parameters).

You can add the middleware manually if you need more control over where they appear in the stacks.

Sidekiq::EncryptedArgs.secret = "YourSecretKey"

Sidekiq.configure_client do |config|
  config.client_middleware do |chain|
    chain.prepend Sidekiq::EncryptedArgs::ClientMiddleware
  end
end

Sidekiq.configure_server do |config|
  config.server_middleware do |chain|
    chain.add Sidekiq::EncryptedArgs::ServerMiddleware
  end

  # Register client middleware on the server so that starting jobs from within
  # another also get encrypted args.
  # https://github.com/mperham/sidekiq/wiki/Middleware#client-middleware-registered-in-both-places
  config.client_middleware do |chain|
    chain.prepend Sidekiq::EncryptedArgs::ClientMiddleware
  end
end

If there is specific middleware you want to insert the encryption or decryption middleware before or after, you can use the following helper methods:

Sidekiq::EncryptedArgs.encrypt_before(SomeOtherMiddlewareClass)
Sidekiq::EncryptedArgs.encrypt_after(SomeOtherMiddlewareClass)
Sidekiq::EncryptedArgs.decrypt_before(SomeOtherMiddlewareClass)
Sidekiq::EncryptedArgs.decrypt_after(SomeOtherMiddlewareClass)

Worker Configuration

To declare that a worker is using encrypted arguments, you must set the encrypted_args sidekiq option.

Setting the option to true will encrypt all the arguments passed to the perform method.

class SecretWorker
  include Sidekiq::Job

  sidekiq_options encrypted_args: true

  def perform(arg_1, arg_2, arg_3)
  end
end

You can also choose to only encrypt specific arguments by specifying either argument names (as symbols or strings) or argument positions (as integers). This is useful to preserve visibility into non-sensitive arguments for troubleshooting or other reasons. All of these examples encrypt just the second argument to the perform method.

# Pass in a single argument name
sidekiq_options encrypted_args: :arg_2
# or as a string
sidekiq_options encrypted_args: "arg_2"
# or as an array
sidekiq_options encrypted_args: [:arg_2]

def perform(arg_1, arg_2, arg_3)
end

# Pass in an integer indicating which argument position should be encrypted (0-indexed)
sidekiq_options encrypted_args: 1
# or as an array
sidekiq_options encrypted_args: [1]

def perform(arg_1, arg_2, arg_3)
end

You don't need to change anything else about your workers. All of the arguments passed to the perform method will already be unencrypted when the method is called.

Rolling Secrets

If you need to roll your secret, you can simply provide an array when setting the secret.

Sidekiq::EncryptedArgs.secret = ["CurrentSecret", "OldSecret", "EvenOlderSecret"]

The first (left most) key will be considered the current key, and is used for encrypting arguments. When decrypting, we iterate over the secrets list until we find the correct one. This allows you to switch you secret keys without breaking jobs already enqueued in Redis.

If you are using the SIDEKIQ_ENCRYPTED_ARGS_SECRET environment variable to specify your secret, you can separate multiple keys with whitespace (spaces or tabs).

You can also safely add encryption to an existing worker. Any jobs that are already enqueued will still run even without having the arguments encrypted in Redis.

Encryption

Encrypted arguments are stored using AES-256-GCM with a key derived from your secret using PBKDF2. For more info on the underlying encryption, refer to the SecretKeys gem.

Installation

Add this line to your application's Gemfile:

gem "sidekiq-encrypted_args"

And then execute:

$ bundle

Or install it yourself as:

$ gem install sidekiq-encrypted_args

Contributing

Open a pull request on GitHub.

Please use the standardrb syntax and lint your code with standardrb --fix before submitting.

License

The gem is available as open source under the terms of the MIT License.