Skip to content

Reserve padding bytes in the metadata - #83

Merged
rwols merged 12 commits into
developfrom
feature/preallocate-large-header
Aug 24, 2026
Merged

Reserve padding bytes in the metadata#83
rwols merged 12 commits into
developfrom
feature/preallocate-large-header

Conversation

@Siebje

@Siebje Siebje commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

This change allows pre-allocating a header of fixed size, which allows a user to make modifications and additions to the global parameters without needing to completely rewrite the file (up to the preset limit). The initial reserved size was 1MB, but it was reduced to 256kB after some user feedback.

close #77

// which is not been finalized in Java 21.
System.gc();
try {
Thread.sleep(100);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the sleep of 100ms needed here?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you considered reading the comments, the commit log, or the method name? The name of the method is 'awaitFileUnmapping'. System.gc() returns immediately, which will lead to the file handle being kept open until the GC is complete.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment above System.gc(); mentions "requires a garbage collect to have been performed". But it does not mention why the sleep(100) is necessary. I guess because System.gc() is asynchronous, which I didn't know.

The docs about System.gc mention:

When control returns from the method call, the Java Virtual Machine has made a best effort to reclaim space from all discarded objects.

This seems to suggest it is not asynchronous?

I suppose it's a request to do GC, but modern GCs are running in background threads.

@rwols rwols Aug 24, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment mentions:

Other fixes required either a Java 8 Cleaner.clean() call not accessible from Java 21

Can this not work for java 9-21 ?

sun.misc.Unsafe.invokeCleaner(ByteBuffer)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and then from java 24+ using arenas...?

try (Arena arena = Arena.ofConfined()) {
    MemorySegment segment = MemorySegment.mapFile(..., arena);
}

Comment thread TRS_FILE_FORMAT_SPECIFICATION.md Outdated
Comment thread TRS_FILE_FORMAT_SPECIFICATION.md Outdated
Comment thread TRS_FILE_FORMAT_SPECIFICATION.md Outdated
Comment thread TRS_FILE_FORMAT_SPECIFICATION.md Outdated
Comment thread TRS_FILE_FORMAT_SPECIFICATION.md Outdated
Comment thread TRS_FILE_FORMAT_SPECIFICATION.md Outdated
Comment thread TRS_FILE_FORMAT_SPECIFICATION.md Outdated
Comment thread TRS_FILE_FORMAT_SPECIFICATION.md Outdated
@Siebje
Siebje force-pushed the feature/preallocate-large-header branch from 143e368 to 39edc87 Compare August 21, 2026 13:35
@rwols rwols changed the title Feature/preallocate large header Preallocate large header Aug 24, 2026
@rwols
rwols merged commit 9a5ed2d into develop Aug 24, 2026
1 check passed
@rwols
rwols deleted the feature/preallocate-large-header branch August 24, 2026 08:49
@rwols rwols changed the title Preallocate large header Reserve padding bytes in the metadata Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pre-allocate a fixed header size

2 participants