fio-plot: 1.1.16 by johnrichardrinehart · Pull Request #435004 · NixOS/nixpkgs

johnrichardrinehart · 2025-08-19T13:59:23Z

Sorry ahead-of-time if this PR is absolute crap. I have little to no experience with python packaging and I have attention issues, so there may be a lot of small things to clean up, here. Thanks for your review.

The goal of this PR is to add fio-plot (a package which provides two binaries, fio-plot and bench_fio), to legacyPackages. I've only tested that the binaries build and seem to run on x86_64-linux:

$ ./result/bin/bench_fio                                                                                                                
　
usage: bench_fio [-h] -d TARGET [TARGET ...] -t {device,file,directory,rbd} [-P CEPH_POOL] [-s SIZE] -o OUTPUT [-b BLOCK_SIZE [BLOCK_SIZE ...]]
                 [--iodepth IODEPTH [IODEPTH ...]] [--numjobs NUMJOBS [NUMJOBS ...]] [--runtime RUNTIME] [-p] [--precondition-repeat]
                 [--precondition-template PRECONDITION_TEMPLATE] [-m MODE [MODE ...]] [--rwmixread RWMIXREAD [RWMIXREAD ...]] [-e ENGINE] [--direct DIRECT] [--loops LOOPS]
                 [--time-based] [--entire-device] [--ss SS] [--ss-dur SS_DUR] [--ss-ramp SS_RAMP] [--extra-opts EXTRA_OPTS [EXTRA_OPTS ...]] [--invalidate INVALIDATE]
                 [--quiet] [--loginterval LOGINTERVAL] [--dry-run] [--destructive] [--remote REMOTE] [--remote-checks] [--remote-timeout REMOTE_TIMEOUT] [--create]
                 [--parallel]
bench_fio: error: the following arguments are required: -d/--target, -t/--type, -o/--output
$ ./result/bin/fio-plot 
usage: fio-plot [-h] -i INPUT_DIRECTORY [INPUT_DIRECTORY ...] [-o OUTPUT_FILENAME] -T TITLE [-s SOURCE] (-L | -l | -N | -H | -g | -C) [--disable-grid] [--enable-markers]
                [--subtitle SUBTITLE] [-d IODEPTH [IODEPTH ...]] [-n NUMJOBS [NUMJOBS ...]] [-M [MAXDEPTH]] [-J [MAXJOBS]] [-D [DPI]] [-p [PERCENTILE]]
                -r {read,write,randread,randwrite,randrw,trim,rw,readwrite,randtrim,trimwrite} [-m MAX_Z] [-e MOVING_AVERAGE] [--min-iops MIN_IOPS] [--min-lat MIN_LAT]
                [-t {bw,iops,lat,slat,clat} [{bw,iops,lat,slat,clat} ...]] [-f {read,write} [{read,write} ...]] [--truncate-xaxis TRUNCATE_XAXIS]
                [--xlabel-depth XLABEL_DEPTH] [--xlabel-parent XLABEL_PARENT] [--xlabel-segment-size XLABEL_SEGMENT_SIZE] [--xlabel-single-column] [-w LINE_WIDTH]
                [--group-bars] [--show-cpu] [--show-data] [--show-ss] [--table-lines] [--max-lat MAX_LAT] [--max-clat MAX_CLAT] [--max-slat MAX_SLAT] [--max-iops MAX_IOPS]
                [--max-bw MAX_BW] [--draw-total] [--colors COLORS [COLORS ...]] [--disable-fio-version] [--title-fontsize TITLE_FONTSIZE]
                [--subtitle-fontsize SUBTITLE_FONTSIZE] [--source-fontsize SOURCE_FONTSIZE] [--credit-fontsize CREDIT_FONTSIZE] [--table-fontsize TABLE_FONTSIZE]
                [--include-hosts INCLUDE_HOSTS [INCLUDE_HOSTS ...] | --exclude-hosts EXCLUDE_HOSTS [EXCLUDE_HOSTS ...]]
fio-plot: error: the following arguments are required: -i/--input-directory, -T/--title, -r/--rw

Now, in order to provide the application package, I also needed to add the corresponding python module package, fio_plot. But, fio_plot depends on https://github.com/Technologicat/pyan/, which is not in tree, yet.

Also, I was running into weird issues around bin files already existing when the fio_plot repository only provided its setup.py. The world seems to be moving toward pyproject.toml so you might notice that I've also bundled in a patch to the source repository which provides a pyproject.toml. This file unblocks the build. I'm going to make a corresponding PR to https://github.com/louwrentius/fio-plot/, but I'm not sure if this project is still maintained. I also want to be able to point to this PR in that PR and say "hey, I needed to hack something up in nixpkgs because your project packaging is stale". I'm happy to remove this patch later if the source repository includes these changes.

So, rather then break this into 3 separate/stacked PRs, I decided to provide a single PR. I hope that's okay.

Things done

Add a 👍 reaction to pull requests you find important.

RossSmyth

Mostly good, a few little changes:

Some sanity checks
Remove with lib;
Use python3Packages rather than python3.pkgs because the latter doesn't splice correctly
Is there an upstream PR for the pyproject.toml? I would recommend putting your patch there if not, then fetching it.

RossSmyth · 2025-08-19T18:49:51Z

pkgs/development/python-modules/fio_plot/default.nix

Suggested change

python,

RossSmyth · 2025-08-19T18:50:02Z

pkgs/development/python-modules/fio_plot/default.nix

Suggested change

python.pkgs.buildPythonPackage rec {

buildPythonPackage rec {

RossSmyth · 2025-08-19T18:50:11Z

pkgs/development/python-modules/fio_plot/default.nix

Suggested change

build-system = [ python.pkgs.setuptools ];

build-system = [ setuptools ];

RossSmyth · 2025-08-19T18:50:21Z

pkgs/development/python-modules/fio_plot/default.nix

Suggested change

dependencies = with python.pkgs; [

dependencies = [

RossSmyth · 2025-08-19T18:50:59Z

pkgs/development/python-modules/fio_plot/default.nix

Suggested change

meta = with lib; {

description = "Create charts from FIO storage benchmark tool output";

homepage = "https://github.com/louwrentius/fio-plot";

license = licenses.bsd3;

maintainers = with maintainers; [ johnrichardrinehart ];

};

meta = {

description = "Create charts from FIO storage benchmark tool output";

homepage = "https://github.com/louwrentius/fio-plot";

license = lib.licenses.bsd3;

maintainers = with lib.maintainers; [ johnrichardrinehart ];

};

RossSmyth · 2025-08-19T18:54:27Z

pkgs/by-name/fi/fio-plot/package.nix

Suggested change

fio_plot = python3.pkgs.fio_plot;

fio_plot = python3Packages.fio_plot;

RossSmyth · 2025-08-19T18:54:38Z

pkgs/by-name/fi/fio-plot/package.nix

Suggested change

python3.pkgs.buildPythonApplication rec {

python3Packages.buildPythonApplication rec {

RossSmyth · 2025-08-19T18:55:01Z

pkgs/by-name/fi/fio-plot/package.nix

Suggested change

build-system = [ python3.pkgs.setuptools ];

build-system = [ python3Packages.setuptools ];

RossSmyth · 2025-08-19T18:55:51Z

pkgs/by-name/fi/fio-plot/package.nix

Or whever it is called

Suggested change

maintainers = with maintainers; [ johnrichardrinehart ];

};

maintainers = with maintainers; [ johnrichardrinehart ];

mainProgram = "fio-plot";

};

RossSmyth · 2025-08-19T18:56:27Z

pkgs/by-name/fi/fio-plot/0001-wip.patch

What's this wip file?

Oh, man, haha. That shouldn't be in the history. It's the exact same patch as in the library package (python-modules/fio_plot/*.patch). I renamed it and pointed the application to the library's patch. Sorry about that.

johnrichardrinehart · 2025-08-19T23:13:20Z

@RossSmyth Thanks for the deep review. I've:

created a PR to fix: add a pyproject.toml to modernize the build system louwrentius/fio-plot#156 and used that commit to pull in my patch and removed my in-tree patch file;
removed with lib; in the various meta sections;
added a pythonImportsCheck for pyan3 (module names itself "pyan", apparently);
removed things like python3Packages.foo and instead moved foo into the args directly;
added a meta.mainProgram to fio-plot;
made sure that it still builds;

I think that's everything. Please let me know if there's more work to do!

RossSmyth · 2025-08-20T00:43:16Z

Hm taking a bit of a closer look, is the python module fio_plot the same thing as the binary fio-plot except run as a program?

RossSmyth

I didn't notice this when I was reviewing earlier. Since it is just a different view of the same source you can just have this.

RossSmyth · 2025-08-20T00:57:39Z

pkgs/by-name/fi/fio-plot/package.nix

Yeah you can just replace the whole thing with this.

Suggested change

{

lib,

fetchFromGitHub,

fio,

python3Packages,

}:

let

fio_plot = python3Packages.fio_plot;

in

python3Packages.buildPythonApplication rec {

pname = "fio-plot";

version = "1.1.16";

pyproject = true;

pythonPath = [

fio_plot

];

patches = fio_plot.patches;

build-system = [ python3Packages.setuptools ];

src = fetchFromGitHub {

owner = "louwrentius";

repo = "fio-plot";

tag = "v${version}";

hash = "sha256-yN0gVm6ZYEIoh91d+0ohJ9yU+VWwYEq3MoG+WgBrs2Q=";

};

dependencies = [

fio

];

meta = {

description = "Create charts from FIO storage benchmark tool output";

homepage = "https://github.com/louwrentius/fio-plot";

license = lib.licenses.bsd3;

mainProgram = "fio-plot";

maintainers = with lib.maintainers; [ johnrichardrinehart ];

};

}

{

python3Packages,

}:

python3Packages.toPythonApplication python3Packages.fio_plot

Nice! I had to overrideAttrs for meta.mainProgram but otherwise I did exactly that.

RossSmyth · 2025-08-20T00:57:57Z

pkgs/development/python-modules/fio_plot/default.nix

Don't forget the import check

RossSmyth · 2025-08-20T00:58:44Z

You also need to squash your commits to follow the commit convention.

johnrichardrinehart · 2025-08-20T02:13:19Z

You also need to squash your commits to follow the commit convention.

Alright. I'll do that if the current state of all the files look good (as of 4558e93e7d269e3750083f03c659637847395294). Let me know and then I'll squash. I kept the history around in case I needed to revert and to support the review.

RossSmyth

Last thing

RossSmyth · 2025-08-20T02:28:28Z

pkgs/by-name/fi/fio-plot/package.nix

You can check the output with nix eval -f ./. fio-plot.meta

Suggested change

meta.mainProgram = "fio-plot";

meta = old.meta // { mainProgram = "fio-plot"; };

Ah, nice. Changed!

johnrichardrinehart · 2025-08-20T02:34:34Z

Last thing

Squashing.

RossSmyth · 2025-08-20T02:38:55Z

Oops, for the commit structure it should be three commits:

python3Packages.pyan3: init at v1.2.0
python3Packages.fio_plot: init at 1.1.16
fio-plot: init at 1.1.16

One for each new package added.

johnrichardrinehart · 2025-08-20T02:51:32Z

Oops, for the commit structure it should be three commits:

python3Packages.pyan3: init at v1.2.0

python3Packages.fio_plot: init at 1.1.16

fio-plot: init at 1.1.16

One for each new package added.

Okay, I think we're stable!

RossSmyth

Nice!

ghost · 2025-08-21T09:13:53Z

pkgs/development/python-modules/pyan3/default.nix

Suggested change

description = "Static call graph generator. The official Python 3 version.";

description = "Static call graph generator";

Be short, just one sentence
Not end with a period
https://github.com/NixOS/nixpkgs/tree/master/pkgs#meta-attributes

ghost · 2025-08-21T09:14:45Z

pkgs/development/python-modules/pyan3/default.nix

please add mainProgram

This is a package. It does not have a mainProgram. At least not how it is used here.

ghost · 2025-08-21T09:15:25Z

pkgs/development/python-modules/pyan3/default.nix

please add pytest

fix: fio_plot tests chore: abbreviate pyan3 description

nix-owners bot requested a review from natsukium August 19, 2025 14:01

johnrichardrinehart force-pushed the fio_plot branch from 69f2713 to b3c7ce6 Compare August 19, 2025 14:14

RossSmyth suggested changes Aug 19, 2025

View reviewed changes

nixpkgs-ci bot added 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 6.topic: python Python is a high-level, general-purpose programming language. labels Aug 19, 2025

johnrichardrinehart mentioned this pull request Aug 19, 2025

fix: add a pyproject.toml to modernize the build system louwrentius/fio-plot#156

Open

johnrichardrinehart requested a review from RossSmyth August 19, 2025 23:13

RossSmyth suggested changes Aug 20, 2025

View reviewed changes

johnrichardrinehart requested a review from RossSmyth August 20, 2025 02:13

RossSmyth suggested changes Aug 20, 2025

View reviewed changes

johnrichardrinehart force-pushed the fio_plot branch from 9594773 to 51e96b7 Compare August 20, 2025 02:35

johnrichardrinehart requested a review from RossSmyth August 20, 2025 02:35

johnrichardrinehart force-pushed the fio_plot branch from 51e96b7 to 0bd7b8d Compare August 20, 2025 02:37

python3Packages.pyan3: init at v1.2.0

92ab18a

johnrichardrinehart force-pushed the fio_plot branch from 0bd7b8d to 5c5facb Compare August 20, 2025 02:51

RossSmyth approved these changes Aug 20, 2025

View reviewed changes

nixpkgs-ci bot added the 12.approvals: 1 This PR was reviewed and approved by one person. label Aug 20, 2025

ghost reviewed Aug 21, 2025

View reviewed changes

johnrichardrinehart added 2 commits September 1, 2025 09:10

python3Packages.fio_plot: init at 1.1.16

3c20fe3

fix: fio_plot tests chore: abbreviate pyan3 description

fio-plot: init at 1.1.16

a477a2c

johnrichardrinehart force-pushed the fio_plot branch from 5c5facb to a477a2c Compare September 1, 2025 16:11

nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Dec 7, 2025

	python.pkgs.buildPythonPackage rec {
	buildPythonPackage rec {

	build-system = [ python.pkgs.setuptools ];
	build-system = [ setuptools ];

-  meta = with lib; {
-    description = "Create charts from FIO storage benchmark tool output";
-    homepage = "https://github.com/louwrentius/fio-plot";
-    license = licenses.bsd3;
-    maintainers = with maintainers; [ johnrichardrinehart ];
-  };
+  meta = {
+    description = "Create charts from FIO storage benchmark tool output";
+    homepage = "https://github.com/louwrentius/fio-plot";
+    license = lib.licenses.bsd3;
+    maintainers = with lib.maintainers; [ johnrichardrinehart ];
+  };

	fio_plot = python3.pkgs.fio_plot;
	fio_plot = python3Packages.fio_plot;

	python3.pkgs.buildPythonApplication rec {
	python3Packages.buildPythonApplication rec {

	build-system = [ python3.pkgs.setuptools ];
	build-system = [ python3Packages.setuptools ];

-{
-  lib,
-  fetchFromGitHub,
-  fio,
-  python3Packages,
-}:
-let
-  fio_plot = python3Packages.fio_plot;
-in
-python3Packages.buildPythonApplication rec {
-  pname = "fio-plot";
-  version = "1.1.16";
-  pyproject = true;
-  pythonPath = [
-    fio_plot
-  ];
-  patches = fio_plot.patches;
-  build-system = [ python3Packages.setuptools ];
-  src = fetchFromGitHub {
-    owner = "louwrentius";
-    repo = "fio-plot";
-    tag = "v${version}";
-    hash = "sha256-yN0gVm6ZYEIoh91d+0ohJ9yU+VWwYEq3MoG+WgBrs2Q=";
-  };
-  dependencies = [
-    fio
-  ];
-  meta = {
-    description = "Create charts from FIO storage benchmark tool output";
-    homepage = "https://github.com/louwrentius/fio-plot";
-    license = lib.licenses.bsd3;
-    mainProgram = "fio-plot";
-    maintainers = with lib.maintainers; [ johnrichardrinehart ];
-  };
-}
+{
+  python3Packages,
+}:
+python3Packages.toPythonApplication python3Packages.fio_plot

	meta.mainProgram = "fio-plot";
	meta = old.meta // { mainProgram = "fio-plot"; };

	description = "Static call graph generator. The official Python 3 version.";
	description = "Static call graph generator";

Uh oh!

Conversation

johnrichardrinehart commented Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Things done

Uh oh!

RossSmyth left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

johnrichardrinehart commented Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RossSmyth commented Aug 20, 2025

Uh oh!

RossSmyth left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RossSmyth commented Aug 20, 2025

Uh oh!

johnrichardrinehart commented Aug 20, 2025

Uh oh!

RossSmyth left a comment

Choose a reason for hiding this comment

Uh oh!

RossSmyth Aug 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

johnrichardrinehart commented Aug 20, 2025

Uh oh!

RossSmyth commented Aug 20, 2025

Uh oh!

johnrichardrinehart commented Aug 20, 2025

Uh oh!

RossSmyth left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RossSmyth Aug 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

johnrichardrinehart commented Aug 19, 2025 •

edited

Loading

johnrichardrinehart commented Aug 19, 2025 •

edited

Loading

RossSmyth Aug 20, 2025 •

edited

Loading

RossSmyth Aug 21, 2025 •

edited

Loading