MD5 batch size too large when files are large? #1587

New Issue

Open

opened 2026-10-05 00:49:13 +02:00 by adam · 0 comments

adam commented 2026-10-05 00:49:13 +02:00

Owner

Copy link

Copy source

I am noting in the JS file batchManagerGlobal.js that the standard batch size for md5 sync's is 1000 files.

This worked really well until I started processing files that were 8MB+. After a few hundred seconds the command would timeout because it wasn't receiving data back fast enough. Is it possible to make the value a bit more dynamic based on file size?

Changing the value temporarily to a size of 10 and it was able to process everything after a few minutes.

I am noting in the JS file ```batchManagerGlobal.js``` that the standard batch size for md5 sync's is 1000 files. This worked really well until I started processing files that were 8MB+. After a few hundred seconds the command would timeout because it wasn't receiving data back fast enough. Is it possible to make the value a bit more dynamic based on file size? Changing the value temporarily to a size of 10 and it was able to process everything after a few minutes. <!-- github-sync-id:1470771693 -->

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

master

issue-2560-people

translation

16.x

feat/embedding-based-contextual-tagging

issue-2576-search-include-ai-in-search-results

feat/search-ocr-ai

issue-2391-update-menu-design

15.x

detect-container

issue-2386-update-filters-users-activity

issue-2362-add-filters-options-in-admin

issue-2367-imagemagick7

issue-2225-quicksearch-help

14.x

issue-2164-BatchManagerUnitMode-Redesign

13.x

12.x

issue-1657-SVG_Support

issue-1703-historySearchMethod

11.x

2.10

2.9

2.8

2.7

2.5

2.6

2.4

2.3

2.2

2.1

2.0

1.7

1.3

1.6

1.5

1.4

16.4.0

16.3.0

16.2.0

16.1.0

16.0.0

16.0.0RC3

16.0.0RC2

15.7.0

16.0.0RC1

16.0.0beta1

16.0.0beta2

15.6.0

15.5.0

15.4.0

15.3.0

15.2.0

15.1.0

15.0.0

15.0.0beta3

15.0.0beta2

15.0.0beta1

14.5.0

14.4.0

14.3.0

14.2.0

14.1.0

14.0.0

14.0.0RC2

14.0.0RC1

14.0.0beta3

14.0.0beta2

13.8.0

14.0.0beta1

13.7.0

13.6.0

13.5.0

13.4.0

13.3.0

13.2.0

13.1.0

13.0.0

13.0.0RC4

13.0.0RC3

13.0.0RC2

13.0.0RC1

12.3.0

13.0.0beta2

13.0.0beta1

12.2.0

12.1.0

12.0.0

12.0.0RC2

12.0.0RC1

12.0.0beta2

12.0.0beta1

11.5.0

11.4.0

11.3.0

11.2.0

11.1.0

11.0.0

11.0.0beta5

2.11.0beta4

2.11.0beta3

2.11.0beta2

2.11.0beta1

2.10.2

2.10.1

2.10.0

2.10.0RC1

2.10.0beta2

2.10.0beta1

2.9.5

2.9.4

2.9.3

2.9.2

2.9.1

2.9.0

2.9.0RC2

2.9.0RC1

2.9.0beta2

2.8.6

2.8.5

2.8.4

2.9.0beta1

2.8.3

2.8.2

2.8.1

2.8.0

2.8.0RC2

2.8.0RC1

2.7.4

2.7.3

2.6.5

2.5.6

2.5.5

2.6.4

2.7.2

2.7.1

2.7.0

2.7.0RC2

2.7.0RC1

2.7.0beta2

2.6.3

2.7.0beta1

2.6.2

2.5.4

2.6.1

2.6.0

2.6.0RC2

2.6.0RC1

2.6.0beta2

2.5.3

2.5.2

2.6.0beta1

2.5.1

2.5.0

2.5.0RC2

2.4.7

2.5.0RC1

2.4.6

2.5.0beta3

2.5.0beta2

2.5.0beta1

2.4.5

2.4.4

2.4.3

2.4.2

2.4.1

2.4.0

2.3.5

2.4.0RC3

2.3.4

2.4.0RC2

2.4.0RC1

2.3.3

2.4.0beta2

2.4.0beta1

2.3.2

2.3.1

2.3.0

2.3.0RC2

2.3.0RC1

2.2.5

2.3.0beta3

2.2.4

2.3.0beta2

2.2.3

2.2.2

2.3.0beta1

2.2.1

2.2.0

2.2.0RC4

2.2.0RC3

2.2.0RC2

2.1.6

2.2.0RC1

2.1.5

2.1.4

2.1.3

2.1.2

2.1.1

2.1.0

2.0.10

2.1.0RC3

2.1.0RC2

2.1.0RC1

2.0.9

2.0.8

2.0.7

2.0.6

2.0.5

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

2.0.0RC4

1.7.3

2.0.0RC3

2.0.0RC2

2.0.0RC1

1.7.2

1.7.1

1.7.0

1.7.0RC2

1.7.0RC1

1.6.2

1.6.1

1.6.0

1.6.0RC2

1.6.0RC1

1.5.2

1.5.1

1.5.0

1.5.0RC2

1.5.0RC1

1.4.1

1.4.0

1.4.0RC3

1.4.0RC2

1.3.4

1.4.0RC1

1.3.3

1.3.2

1.3.1

1.3.0

1.3.0RC2

1.3.0RC1

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

adam (Adam Wojcieszonek)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: adam/Piwigo#1587