Compare commits

...

374 Commits

Author SHA1 Message Date
Fabian Thomys
a87e661745 Debugger/Qt: Unlock debugger table view layout 2025-07-04 14:49:41 -04:00
JordanTheToaster
f750eb56ab 3rdparty: Update rcheevos to v12.0.0 2025-07-04 14:49:13 -04:00
JordanTheToaster
118b604a66 3rdparty: Update fmt to v11.2.0 2025-07-04 14:49:04 -04:00
JordanTheToaster
cfd9343230 Deps: Update shaderc to 2025.3 2025-07-04 14:48:55 -04:00
JordanTheToaster
8bcc2c94b9 3rdparty: Update fast_float to v8.0.2 2025-07-04 14:48:42 -04:00
JordanTheToaster
377930d004 3rdparty: Update rapidyaml to v0.9.0 2025-07-04 14:48:42 -04:00
chaoticgd
a9526b7cc1 Docs: Add missing third party license for Zycore 2025-07-04 14:48:29 -04:00
chaoticgd
ff01a6359b 3rdparty: Update Zydis to v4.1.1 2025-07-04 14:48:29 -04:00
chaoticgd
6c3ea29157 R3000A: Fix DUMP_BLOCKS option 2025-07-04 14:48:29 -04:00
KamFretoZ
36be1a8f7d FSUI: Fix default to game list behaviour 2025-07-04 14:47:19 -04:00
refractionpcsx2
691c3764ac GS/TC: Check channel overlap for local mem invalidation/readbacks 2025-07-04 19:14:46 +02:00
refractionpcsx2
aa0292ada1 GS/TC: Improve dirty overlap conditions 2025-07-04 19:14:46 +02:00
PCSX2 Bot
e4429527e3 [ci skip] Qt: Update Base Translation. 2025-07-04 13:56:06 +02:00
Mrlinkwii
f06285e3a4 UI : fix some wrong spellings 2025-07-03 16:58:07 +02:00
TellowKrinkle
6d47cefefd MacOS: Add gsdumps, save states, and elfs as supported file types 2025-07-03 10:31:45 -04:00
TellowKrinkle
fe71fc6a30 Qt: Support macOS file open events 2025-07-03 10:31:45 -04:00
Gd7
0e649bc70b GameDB: Final Fantasy X fixes.
Set VU0 Clamping Mode to Extra, which fixes the flickering issue with Yojimbo's cloak when EE Clamping Mode is set to Full, without causing any other issues.
Added missing settings to all other versions of FFX, along with fixing and clarifying the comments.
Removed settings for "Final Fantasy X [Bonus Disc - Beyond Final Fantasy]," as it is just a bonus DVD and not an actual game.
2025-07-02 18:28:34 +02:00
weirdbeardgame
e0915d8372 FSUI: Change to constant string literal 2025-07-02 18:27:05 +02:00
PCSX2 Bot
3f4d9cda85 [ci skip] PAD: Update to latest controller database. 2025-06-30 18:01:37 +02:00
PCSX2 Bot
4e25988961 [ci skip] Qt: Update Base Translation. 2025-06-29 20:19:41 -04:00
JordanTheToaster
b1502c754a Deps: Update Windows and Linux to Qt 6.9.1 2025-06-29 19:47:09 -04:00
SternXD
303cef78ed FSUI: Add Save State Incompatible Warning
Signed-off-by: SternXD <stern@sidestore.io>
2025-06-29 17:02:45 -04:00
Davide Pesavento
a72f282156 Qt: Fix minor typos in settings help text 2025-06-29 17:01:25 -04:00
escape209
32e77a2bd3 Fix some timestamps in GameList using wrong locale 2025-06-29 16:57:05 -04:00
KamFretoZ
a53fdfae82 FSUI: Add option to open Game List directly on startup 2025-06-29 16:56:01 -04:00
Haisom
806ab5ad01 Refactor: Renames help function for consistency
- Renamed <IsMemoryCardFolder> to <FileMcd_IsFolder> for better consistency and clarity on MemoryCardFile.cpp.
2025-06-29 16:54:49 -04:00
chaoticgd
f5ddc0d7c6 Debugger: Add Follow Address option in memory view 2025-06-29 16:54:27 -04:00
chaoticgd
a6eb5d32d8 Debugger: Make some more strings translatable 2025-06-29 16:53:22 -04:00
RedDevilus
b03c982706 3rdparty: Soundtouch bump to v2.4.0
Author also moved to https://codeberg.org/soundtouch/soundtouch. Audiochannel bump from 16 to 32 limit.
2025-06-29 16:52:37 -04:00
Ziemas
7835ebd14f debugger: don't refresh thread/module when paused 2025-06-29 16:52:17 -04:00
Ziemas
fd67049cf9 debugger: sanity check module list 2025-06-29 16:52:17 -04:00
Ziemas
c2f05371e4 debugger: update thread list on refresh 2025-06-29 16:52:17 -04:00
Ziemas
5c9c071086 debugger: set thread list vertical resize 2025-06-29 16:52:17 -04:00
Ziemas
f7677105ed debugger: cache thread list in model 2025-06-29 16:52:17 -04:00
Ziemas
9e2a73c7a7 debugger: use monospace font for thread list 2025-06-29 16:52:17 -04:00
Ziemas
c9f5f03d02 debugger: update module list on Refresh event 2025-06-29 16:52:17 -04:00
Ziemas
c313736334 debugger: set vertical resize of module list 2025-06-29 16:52:17 -04:00
Ziemas
1eb9a60b4a debugger: cache module list in model 2025-06-29 16:52:17 -04:00
Ziemas
e50fe50daf debugger: display iop module list 2025-06-29 16:52:17 -04:00
Ziemas
643c83c2a0 debugger: Find and build IOP module list 2025-06-29 16:52:17 -04:00
JordanTheToaster
44733d7655 3rdparty: Update Vulkan-Headers to v1.4.320 2025-06-29 16:51:05 -04:00
JordanTheToaster
fcdd43fc78 3rdparty: Update vkmemoryallocator to 3.3.0 2025-06-29 16:51:05 -04:00
JordanTheToaster
78643623c3 3rdparty: Update d3d12memalloc to 3.0.1 2025-06-29 16:51:05 -04:00
SternXD
b27846ce39 FSUI: Add custom RetroAchievements login dialog
FSUI: Add custom RetroAchievements login dialog
Signed-off-by: SternXD <stern@sidestore.io>
2025-06-29 16:50:14 -04:00
TellowKrinkle
b87b8eb7bb MacOS: Add pnach definition to Info.plist
Ensures macOS treats it as a known file type, so it doesn't try to prevent users from adding it as a file extension
2025-06-29 16:47:22 -04:00
dependabot[bot]
709dfdd3f2 Bump the ci-deps group across 1 directory with 2 updates
Bumps the ci-deps group with 2 updates in the / directory: [softprops/action-gh-release](https://github.com/softprops/action-gh-release) and [baptiste0928/cargo-install](https://github.com/baptiste0928/cargo-install).


Updates `softprops/action-gh-release` from 2.2.2 to 2.3.2
- [Release notes](https://github.com/softprops/action-gh-release/releases)
- [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md)
- [Commits](da05d55257...72f2c25fcb)

Updates `baptiste0928/cargo-install` from 3.3.0 to 3.3.1
- [Release notes](https://github.com/baptiste0928/cargo-install/releases)
- [Changelog](https://github.com/baptiste0928/cargo-install/blob/main/CHANGELOG.md)
- [Commits](91c5da1557...e38323ef01)

---
updated-dependencies:
- dependency-name: softprops/action-gh-release
  dependency-version: 2.3.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: ci-deps
- dependency-name: baptiste0928/cargo-install
  dependency-version: 3.3.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: ci-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-06-29 16:46:44 -04:00
TheLastRar
faef28d5c4 Deps: Also force PDB files for Qt's plugins 2025-06-29 16:45:55 -04:00
TheLastRar
309dd4f847 Deps: Correct comment 2025-06-29 16:45:55 -04:00
TheLastRar
506897fab2 Build: Copy force enabled symbols
Currently only SDL and Qt
2025-06-29 16:45:55 -04:00
SternXD
4e0aff053c SaveState: Improve version incompatibility error message
Make the error message clearer by showing which version of PCSX2 created
the save state and which version is needed to recover it.

Signed-off-by: SternXD <stern@sidestore.io>
2025-06-29 16:45:10 -04:00
Florin9doi
2ba6fbbcc2 USB: Connect the GT Force pedals 2025-06-29 16:44:28 -04:00
TheLastRar
e987eac545 CI/Appimage: Simplify how we bundle Wayland plugins 2025-06-29 16:44:07 -04:00
Florin9doi
6b9781fee6 USB: Throttle the Buzz data packets 2025-06-29 16:43:37 -04:00
TJnotJT
181ceb38cc GS: Put psm_str in the GSUtil struct. 2025-06-29 16:43:09 -04:00
chaoticgd
595ce0ea19 Qt: Add security warning to setup wizard 2025-06-29 16:41:56 -04:00
TheLastRar
b624330155 Deps: Update Pluto(S)VG
PlutoVG is updated to v1.1.0
PlutoSVG is updated to v0.0.7
2025-06-29 16:41:19 -04:00
JordanTheToaster
380c316869 Deps: Update SDL3 3.2.16 2025-06-29 16:40:27 -04:00
Florin9doi
31dacc2d21 Update Negcon to support Wipeout Fusion 2025-06-29 16:40:17 -04:00
Ty
b557a82009 [ci skip] 2.5.x Development Cycle 2025-06-29 16:34:51 -04:00
Ty
e4af1c4244 [ci-skip] 2.4 Release 2025-06-29 15:47:12 -04:00
JordanTheToaster
7c26ac5578 GameDB: Various fixes 2025-06-29 21:07:07 +02:00
JordanTheToaster
2948d50b0d GameDB: Shadow Hearts fixes 2025-06-29 10:32:13 +02:00
lightningterror
48fefddcb2 GS/HW: Fix -Wunused-variable warnings. 2025-06-29 10:17:57 +02:00
lightningterror
f4d8af2f0d GS/GL: Don't re enable blend after SetupDATE.
Redundant calls, blending will be enabled if needed.
2025-06-29 10:17:57 +02:00
lightningterror
afa3108623 GS/DX11: Adjust blend/depth stencil state.
OMSetDepthStencilState:
If state is nullptr, new state is also nullptr but stencil ref changes no need to set a new OMSetDepthStencilState.

OMSetBlendState:
If state is nullptr, new state is also nullptr but blend factor changes no need to set a new OMSetBlendState.
2025-06-29 10:17:57 +02:00
lightningterror
6fcfddf19a GS/GL/DX11: Resolve potential rtv/srv conflict for primid date.
primid_texture texture was already bound to rtv, so we either need to unbind it first from rtv, or we can bind it to srv after rtv bind has been changed.

Note: DX11 is fine since gpu state will be updated when DrawIndexedPrimitive is executed but it's nice to keep code parity and visual a visual state
what the pipeline order should do.
2025-06-29 10:17:57 +02:00
lightningterror
6d2f442cbd GS/DX11: Backports the avoid framebuffer optimization from GL.
Unlike OpenGL, we don't need to worry about adding extra barriers here
since we already do fb copies so rt tex won't be bound as an rtv and srv
at the same time, we also check for conflicts beforehand.
2025-06-29 10:17:57 +02:00
PCSX2 Bot
e8260e7191 [ci skip] Qt: Update Base Translation. 2025-06-28 20:04:49 -04:00
Ty
48a4367a2c New translations pcsx2-qt_en.ts (Spanish, Latin America)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
edd3540f34 New translations pcsx2-qt_en.ts (Guarani)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
111c32c5ed New translations pcsx2-qt_en.ts (Quechua)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
af22ca3ae0 New translations pcsx2-qt_en.ts (Hindi)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
5e2caa326c New translations pcsx2-qt_en.ts (Latvian)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
50f9f60341 New translations pcsx2-qt_en.ts (Croatian)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
65f61b143a New translations pcsx2-qt_en.ts (Persian)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
10f4892b7e New translations pcsx2-qt_en.ts (Indonesian)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
f71d518f75 New translations pcsx2-qt_en.ts (Portuguese, Brazilian)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
c5b11fe484 New translations pcsx2-qt_en.ts (Vietnamese)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
068fd04f15 New translations pcsx2-qt_en.ts (Chinese Traditional)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
a461750b55 New translations pcsx2-qt_en.ts (Chinese Simplified)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
291df6a0e4 New translations pcsx2-qt_en.ts (Ukrainian)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
8afeb4ea61 New translations pcsx2-qt_en.ts (Turkish)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
b30f246900 New translations pcsx2-qt_en.ts (Swedish)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
7fd042b197 New translations pcsx2-qt_en.ts (Serbian (Cyrillic))
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
b6b3e364f2 New translations pcsx2-qt_en.ts (Russian)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
70e5f18971 New translations pcsx2-qt_en.ts (Portuguese)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
1fb35593d5 New translations pcsx2-qt_en.ts (Polish)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
b1b9c32644 New translations pcsx2-qt_en.ts (Norwegian)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
8eeed7d42c New translations pcsx2-qt_en.ts (Dutch)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
584493d945 New translations pcsx2-qt_en.ts (Lithuanian)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
da8e551e77 New translations pcsx2-qt_en.ts (Korean)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
a717930d5a New translations pcsx2-qt_en.ts (Georgian)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
abc000f614 New translations pcsx2-qt_en.ts (Japanese)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
cd51f1def3 New translations pcsx2-qt_en.ts (Italian)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
f5573cf0ab New translations pcsx2-qt_en.ts (Hungarian)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
1d036a4897 New translations pcsx2-qt_en.ts (Hebrew)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
f9eced6244 New translations pcsx2-qt_en.ts (Finnish)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
90a7253c66 New translations pcsx2-qt_en.ts (Greek)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
31e065f83a New translations pcsx2-qt_en.ts (German)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
571f443339 New translations pcsx2-qt_en.ts (Danish)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
4a4157919c New translations pcsx2-qt_en.ts (Czech)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
e73aa0e81b New translations pcsx2-qt_en.ts (Catalan)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
2343246315 New translations pcsx2-qt_en.ts (Bulgarian)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
e5f41ce175 New translations pcsx2-qt_en.ts (Arabic)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
8184f2eaa9 New translations pcsx2-qt_en.ts (Afrikaans)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
e0bb465945 New translations pcsx2-qt_en.ts (Spanish)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
55749a63eb New translations pcsx2-qt_en.ts (French)
[ci skip]
2025-06-28 13:31:32 -04:00
Ty
aa1b8db3ea New translations pcsx2-qt_en.ts (Romanian)
[ci skip]
2025-06-28 13:31:32 -04:00
JordanTheToaster
023713fd67 GameDB: The Mummy fixes 2025-06-28 13:05:00 -04:00
refractionpcsx2
35093c3e37 GS/HW: Avoid single pixel overlap on lookups + improve buffer splitting 2025-06-27 18:04:29 +02:00
refractionpcsx2
4e5dac3e25 GameDB: Add fixes for MLB 11 and Toro to Kyuujitsu 2025-06-27 18:03:48 +02:00
refractionpcsx2
c52e84ac41 GS/HW: Add ability to detect 16bit clears with 32bit draw 2025-06-27 18:03:48 +02:00
refractionpcsx2
7f8488771d GS/TC: Improve heuristics for sequenced draws inside targets 2025-06-27 18:03:48 +02:00
refractionpcsx2
57ff271f4b GS/HW: Fix up offset Z behaviours and copy ranges 2025-06-27 18:03:48 +02:00
refractionpcsx2
43703755f8 GS/HW: Improve double half clear detection heuristics 2025-06-27 18:03:48 +02:00
refractionpcsx2
17b6cc00ab GS/HW: When expanding a target for display, expand valid area 2025-06-27 18:03:19 +02:00
JordanTheToaster
a03563b366 GameDB: The Golden Compass fixes 2025-06-26 13:44:29 +02:00
lightningterror
ff9da17498 GS/DX11: Allow to pick whenever to update sr or ss, and some reordering.
No need to call sampler update when updating conflicting srvs.
2025-06-26 13:43:51 +02:00
lightningterror
722bc94270 GS/DX11: Cache shader resource and sampler.
Might help speed things up, requires srv and rtv conflicts to be resolved.
2025-06-26 13:43:51 +02:00
lightningterror
d51a5db5b1 GS/GL: Add missing texture barrier count. 2025-06-26 09:20:54 +02:00
PCSX2 Bot
04541ae2ab [ci skip] Qt: Update Base Translation. 2025-06-25 07:15:25 +02:00
JordanTheToaster
c58a67815b GameDB: Stuart Little 3 loading screen fix 2025-06-25 00:03:28 +02:00
JordanTheToaster
085f964cd9 CI/Linux: Link ffmpeg for the appimage 2025-06-24 06:18:38 +02:00
Mrlinkwii
a8b6e448eb GameDB: remove reference to old GSC 2025-06-24 06:13:14 +02:00
PCSX2 Bot
78ab8381d9 [ci skip] PAD: Update to latest controller database. 2025-06-24 06:12:26 +02:00
PCSX2 Bot
49a5d82086 [ci skip] Qt: Update Base Translation. 2025-06-24 06:12:04 +02:00
TheLastRar
4ed129ccac FSUI: Improve layout scaling in games list selected preview 2025-06-24 06:11:49 +02:00
lightningterror
ed09dca17e GS/DX11: Also check DrawMultiStretchRects copy for srv conflicts with rtv.
Also adjust the naming.
2025-06-23 15:09:03 +02:00
TheLastRar
e64fbb2f0e FSUI: Centre disc/exe icon in game list selected preview
The ImGui::Image path was missing the centring logic when drawing svg icons.
Covers where already correctly centred.
2025-06-23 15:07:56 +02:00
lightningterror
e37f8a6521 IopBios: Fix -Wcompare warnings. 2025-06-23 10:18:00 +02:00
lightningterror
b54bcc0e20 FullscreenUI: Fix -Wunused-variable warnings. 2025-06-23 10:18:00 +02:00
lightningterror
db37d51bb1 GS: Silence 3rdparty vk_mem_alloc warnings.
-Wunused-private-field
2025-06-23 10:18:00 +02:00
lightningterror
197b9fc560 GS/HW: Check if primid texture exists instead if it's a primid draw.
Makes local testing easier to null out primid tex.
Also match naming between gl and dx11.
Change some longs to warning level.
2025-06-23 09:59:22 +02:00
lightningterror
fd30b00205 GS/DX11: Make sure no SRVs are bound using the same texture before binding it to a RTV.
Fixes api hazard warnings.
2025-06-23 09:59:22 +02:00
TheLastRar
f182379d24 FSUI: Don't attempt to translate input profile names in game properties 2025-06-23 09:53:25 +02:00
refractionpcsx2
32ba08980d GS: Treat Q == 0 in STQ as FLT_MIN 2025-06-23 09:46:03 +02:00
PCSX2 Bot
1e1f08abb9 [ci skip] Qt: Update Base Translation. 2025-06-23 06:28:14 +02:00
refractionpcsx2
cbe20c0eed GS/HW: Fix dirty check on SWPrimRenderer check 2025-06-23 06:27:41 +02:00
refractionpcsx2
ec288ffa62 GS/TC: Fix CanTranslate rect BWs, disallow block inside target lookup 2025-06-23 06:27:41 +02:00
JordanTheToaster
7e18c02c7e GameDB: Xenosaga Episode III fixes 2025-06-23 06:27:13 +02:00
twingofan
4a505cc239 GameDB: Add memcardFilters to Armored Core: Last Raven NTSC-U SLUS-21338. (#12891) 2025-06-23 06:26:16 +02:00
Kuan-Wei Chiu
35c81106c6 Fix invalid comparator in FullscreenUI game list sort
The game list comparator previously used >= when sorting by CRC in
descending order. This violates the requirements of a strict weak
ordering, as defined by the C++ standard, which mandates that the
comparator must be irreflexive: comp(x, x) must always return false.

Using >= causes comp(a, b) and comp(b, a) to both return true when
a == b, which breaks the irreflexivity and can invalidate the
assumptions made by std::sort. This may lead to undefined behavior,
including memory corruption or segmentation faults.

Replace >= with > to ensure the comparator satisfies the strict weak
ordering requirement, restoring correctness and stability in game list
sorting behavior.
2025-06-22 14:44:34 -04:00
Ziemas
1094222d3f Debugger: Don't check frame count in isAlive 2025-06-22 20:19:13 +02:00
TheLastRar
065d0db4c9 FSUI: Replace various icons with SVG files
Co-Authored-By: KamFretoZ <14798312+kamfretoz@users.noreply.github.com>
2025-06-22 20:15:03 +02:00
JordanTheToaster
bc51537080 GS/HW: Purge Hitman Blood Money CRC 2025-06-22 20:14:09 +02:00
JordanTheToaster
b7dfcb282b ImGuiOverlays: Minor OSD text changes 2025-06-22 20:14:09 +02:00
JordanTheToaster
78a9411766 GameDB: Various Fixes Part 5 2025-06-22 20:14:09 +02:00
TellowKrinkle
4724f67596 GS: Min alpha for AA1 is 0, not 128 2025-06-22 20:13:15 +02:00
JordanTheToaster
e0851bb86f [ci skip] Github: Update app bug report OS list 2025-06-21 22:00:10 -04:00
JordanTheToaster
08e68d9563 GitHub: Update bug report OS list 2025-06-21 15:50:02 -04:00
refractionpcsx2
0b6dccae51 GS/HW: Resize validity to draw size when most of target is covered 2025-06-21 05:52:50 +02:00
refractionpcsx2
60adfd5046 GS/TC: Fix region area on new source interpreted for shuffle 2025-06-21 05:52:50 +02:00
chaoticgd
3e782d355d Docs: Add missing third party license for KDBindings 2025-06-20 22:22:51 +02:00
PCSX2 Bot
22a7324b69 [ci skip] Qt: Update Base Translation. 2025-06-20 02:02:47 +02:00
refractionpcsx2
f3b4c50909 GS/HW: Improve Native Scaling detection + Include direct mem reads 2025-06-20 02:02:30 +02:00
lightningterror
a45f27e6e9 GS/DX: Don't output color for datm/stencil date shaders.
RTV(Render target view) is not bound so no need
to output anything, just clip/discard if needed.

Fixes dx11 api warning that pixel shader writes color
output but no rtv is bound.
2025-06-20 02:01:47 +02:00
lightningterror
2f84bf0cca GS/TC: Don't enable Frame buffer conversion on PSMT8.
We have dedicated shader now, no need.
2025-06-20 02:01:28 +02:00
refractionpcsx2
3e2c3e5075 GS/TC: Slight fix for rect translation with block offset 2025-06-20 02:00:51 +02:00
refractionpcsx2
154259d0a6 GS/HW: Don't resize target if only writing to alpha when RGB is valid 2025-06-20 02:00:51 +02:00
TellowKrinkle
7d2d05ff59 GS: Send all frames to capture, including skipped ones
You wanted those frames recorded right?  Also if you skip them you'll desync audio and everything will be sad.
2025-06-18 20:08:16 -04:00
TheLastRar
f7c587c9a8 CI: Restore repository check in Flathub publish action 2025-06-18 09:07:54 -04:00
PCSX2 Bot
9162b30b18 [ci skip] Qt: Update Base Translation. 2025-06-18 07:09:06 +02:00
chaoticgd
14cfe37c8b Debugger: Fix crash when breakpoint is hit before createMenuBar call 2025-06-17 04:39:32 +02:00
PCSX2 Bot
a27a0ab49d [ci skip] PAD: Update to latest controller database. 2025-06-17 04:38:20 +02:00
Ty
0742fe07ec [ci skip] Testing flathub versioning, please ignore 2025-06-15 14:53:55 -04:00
Ty
65a92470d3 CI: Fix flathub versioning on tagless commits 2025-06-15 14:26:08 -04:00
PCSX2 Bot
10dc1a2da2 [ci skip] Qt: Update Base Translation. 2025-06-14 21:50:20 +02:00
JordanTheToaster
aad2128c32 GSRunner: Fix broken compilation 2025-06-14 21:50:08 +02:00
refractionpcsx2
2640678cac GS/HW: Adjust which function gets used for clearing depth on perfect match 2025-06-14 01:30:00 +02:00
KamFretoZ
8b90e2d53d BPM: Fix CTD when exiting BPM in No GUI mode 2025-06-14 01:28:49 +02:00
KamFretoZ
f87175dc7f Host: Move Batch/NoGUI mode check to Host
Moved from QtHost to Host for more accesibility
2025-06-14 01:28:49 +02:00
refractionpcsx2
b30444c0d0 GS/HW: Fix bug and improve inside target tracking on HW moves 2025-06-12 23:01:36 +01:00
refractionpcsx2
c1da0caf15 GS/HW: Fix possible null reference causing a crash 2025-06-12 18:26:15 +01:00
lightningterror
c4a8fc5b71 Qt: Fix Winconsistent-missing-override warning. 2025-06-11 23:17:53 +02:00
GovanifY
8eb46b5a4c IopBios: remove clang deprecated carveout
we use snprintf now, so this isn't useful anymore.
2025-06-10 20:59:25 +02:00
GovanifY
8be16d1039 IopBios: do not overflow snprintf tmp buffer
We could otherwise overflow as snprintf does not return the number of
written bytes but the number of written bytes assuming an infinite
buffer.
2025-06-10 20:59:25 +02:00
refractionpcsx2
76e6208d1b GS/TC: Fix region for tex in rt depth 2025-06-10 20:12:41 +02:00
refractionpcsx2
bc09080ba5 GS/TC: Remove old inside check from source lookup 2025-06-10 20:12:41 +02:00
refractionpcsx2
a6eb257a3a GS/HW: Allow RT in RT offset on Z buffer if FRAME is disabled 2025-06-10 20:11:32 +02:00
PCSX2 Bot
e62450d255 [ci skip] PAD: Update to latest controller database. 2025-06-10 20:10:22 +02:00
GovanifY
1aa922f700 IopBios: allow %u string formatting in IOP kprintf 2025-06-10 14:47:27 +02:00
GovanifY
4c9d2f99b1 IopBios: truncate printf output if bigger than our buffer
I also increased the buffer size while we are at it to avoid breakage,
despite the obvious unintended allowed situations that we allowed up
until now

Reported-By: Michael Lappas
2025-06-10 14:47:27 +02:00
Ty Lamontagne
f6e899b570 Debugger: Update search results when we are _not_ going to remove them 2025-06-08 15:44:56 -04:00
refractionpcsx2
695c39fba2 GS/HW: Remove channel shuffle override from Namco CRC hack 2025-06-08 16:05:24 +02:00
refractionpcsx2
e5616cff98 GS/HW: Improve shuffle detection robustness 2025-06-08 13:51:19 +02:00
refractionpcsx2
76d5994c1e GS/TC: Improve rect block offset calculation for translation and invalidation 2025-06-08 13:51:19 +02:00
refractionpcsx2
cbb40832a1 GS/HW: Correct block offset target usage 2025-06-08 13:51:19 +02:00
refractionpcsx2
e4bdcde1ca GS/HW: Remove CRC hacks for Kunoichi and Sakura Wars 2025-06-08 13:51:19 +02:00
refractionpcsx2
863e119ff4 GS/HW: Allow conversion of colour to Z formats during HW move 2025-06-08 13:51:19 +02:00
refractionpcsx2
2ccf6dc872 GS: Add IsPageAlignedMasked to return an alignment mask 2025-06-08 13:51:19 +02:00
refractionpcsx2
24ebf1b4f1 GS/HW: Fix bug in target preloading causing misaligned base addresses 2025-06-08 13:51:19 +02:00
refractionpcsx2
ed2832434c GS/HW: Don't look up block offset targets on Exact target lookup 2025-06-08 13:51:19 +02:00
TJnotJT
5e160fca8f Tools: Small fix to gsrunner args. 2025-06-06 18:05:37 -04:00
JordanTheToaster
a0ef82e221 AudioStream: Fix typo with default output latency 2025-06-06 18:03:29 -04:00
icup321
3e7ac3d66c GameDB: Add native scaling to Scooby-Doo! Unmasked 2025-06-06 23:09:37 +02:00
JordanTheToaster
730e6fa737 GameDB: Simple 2000 Vol 92 Fixes 2025-06-06 23:09:26 +02:00
PCSX2 Bot
cdfcd9fddd [ci skip] Qt: Update Base Translation. 2025-06-06 02:31:08 +02:00
refractionpcsx2
b6930c10b9 GS/HW: Clean up target download formats 2025-06-05 13:51:21 +02:00
refractionpcsx2
852734580d GameDB: Update Harry Potter fixes. 2025-06-05 13:51:21 +02:00
refractionpcsx2
d1dc6a9c1d GS/HW: Add 16bit to 8bit conversion shader 2025-06-05 13:51:21 +02:00
Ty
4fa6d3ed3f GSRunner: Add type to shutdown message code 2025-06-04 20:25:32 -04:00
TellowKrinkle
92e190ad6c GSRunner: Fix surfaceless run on macOS 2025-06-04 20:25:32 -04:00
TellowKrinkle
da4fcffef4 Input: Fix crash when shutting down without initializing input 2025-06-04 20:25:32 -04:00
TellowKrinkle
1a5731dd8e MacOS: Better handle directories of non-bundle applications 2025-06-04 20:25:32 -04:00
TellowKrinkle
e764c5cd4e GSRunner: macOS support 2025-06-04 20:25:32 -04:00
TellowKrinkle
e23b247947 GSRunner: Use separate CPU thread 2025-06-04 20:25:32 -04:00
PCSX2 Bot
3d7792436f [ci skip] Qt: Update Base Translation. 2025-06-04 20:03:38 -04:00
chaoticgd
d8187fbea4 Deps: Specify minimum version of KDDockWidgets 2025-06-04 20:02:02 -04:00
chaoticgd
02259ad0a5 Debugger: Add include required for older versions of KDDockWidgets 2025-06-04 20:02:02 -04:00
TellowKrinkle
220a68df9a GS: Warn on texture replacement folder with wrong case 2025-06-04 19:58:41 -04:00
TellowKrinkle
2ced24f69e GS: Create texture dump directory if it doesn't exist 2025-06-04 19:58:41 -04:00
TellowKrinkle
ec91d0dc74 GS: Formatting 2025-06-04 19:58:41 -04:00
TheLastRar
46874f4673 Qt: Add workaround for incorrectly tinted icons after theme switch 2025-06-04 19:39:18 -04:00
TheLastRar
9eac47dc6c Qt: Fix selected gamelist icons being wrong colour after theme switch 2025-06-04 19:39:18 -04:00
RedDevilus
9e3fd5c2e0 CI: Fix flatpak
Try to fix build failing
2025-06-04 20:19:46 +01:00
refractionpcsx2
ae4be6e2b1 GS/CRC: Remove CRC for Simple 2000 Series Vol. 114, update GameDB Fixes 2025-06-04 18:35:00 +02:00
refractionpcsx2
434df49a7d GS/HW: Clear matched target on HW Move 2025-06-04 18:35:00 +02:00
JordanTheToaster
c939c0fcd5 GameDB: Re-add CRC to Death by Degrees 2025-06-04 18:33:36 +02:00
Berylskid
952c39f324 GameDB: Remove SoftwareRendererFMVHack from Armored Core 2 2025-06-03 13:44:40 +02:00
PCSX2 Bot
0fea7e2a70 [ci skip] Qt: Update Base Translation. 2025-06-03 03:40:13 +02:00
lightningterror
c1baab68d0 GS: Better handle hazards when dx12/vk device creation fails.
VK/DX12: Move CreateNullTexture before reading shader resource.
Fixes null pointer dereference.

VK: Check if vertex buffer is valid before binding.
Fixes vertex buffer validation error null handle.
2025-06-02 21:28:41 +02:00
PCSX2 Bot
ccef18f7a9 [ci skip] PAD: Update to latest controller database. 2025-06-02 19:08:36 +02:00
TheLastRar
8cb056bde3 Qt: Use DevicePixelRatioChange to detect DPR changes 2025-06-02 14:58:00 +02:00
TheLastRar
6ecaaee9e0 QtUtils: Remove redundant method 2025-06-02 14:58:00 +02:00
TheLastRar
c5f916bda0 Qt: Fix DPI icon scaling in various settings windows 2025-06-02 14:58:00 +02:00
TheLastRar
52a9a4649c Qt: Improve handling of DPI 2025-06-02 14:58:00 +02:00
refractionpcsx2
d2219b4dbd GS/TC: On RT->Z dst_match delete on format change if not a shuffle 2025-06-02 14:52:46 +02:00
refractionpcsx2
155f603245 GS/HW: Don't make new scaled targets on shuffles if source is downscaled 2025-06-02 14:52:46 +02:00
refractionpcsx2
cb7630a6ab GS/HW: Create new targets on shuffles when no target found
And get rid of some valid sizing thing which is just insane
2025-06-02 14:52:46 +02:00
refractionpcsx2
284fba1ce3 GS/TC: Don't allow Tex in RT 8bit textures from C24
C24 has no alpha channel, 8bit requires all.  This is still allowed for shuffles as that depends which channel it's reading.
2025-06-02 14:52:46 +02:00
refractionpcsx2
aa4bd6c88c GS/HW: Replace frame target if dirty data matches old format. 2025-06-02 14:52:46 +02:00
refractionpcsx2
623993930b GS/HW: Adjust depth size on clear if overlapping by 1 pixel
This is developer misunderstanding of how to use the SCISSOR register, which adds 1 on to the value when it's used, but they put 512x512, when they wanted 512x512.
2025-06-02 14:52:46 +02:00
refractionpcsx2
dbf2c854c6 GS/TC: Adopt valid rgb/alpha from preload merged targets 2025-06-02 14:52:46 +02:00
refractionpcsx2
25351bc05c GS/TC: Correct valid area checks for target combining 2025-06-02 14:52:46 +02:00
refractionpcsx2
27cc5f499c GS/HW: Fix up alpha blending checks 2025-06-02 14:52:46 +02:00
refractionpcsx2
b7c2f39a17 GS/TC: Further matching parameters on preload and tex in rt 2025-06-02 14:52:46 +02:00
refractionpcsx2
3541c1ccf8 GS/TC: Simplify and improve P8 texture conversion inside target 2025-06-02 14:52:46 +02:00
refractionpcsx2
7a05738d11 GS/TC: Allow matching on source if TEX == RT 2025-06-02 14:52:46 +02:00
refractionpcsx2
686220ae0c GS/TC: Improve copying of dst matched data 2025-06-02 14:52:46 +02:00
refractionpcsx2
cf380d36b9 GS/TC: Fix inside target alignment check for ExactTarget lookup 2025-06-02 14:52:46 +02:00
lightningterror
50bc0193ac Core: Bump savestate version.
[SAVEVERSION+]
2025-06-02 01:30:50 +02:00
refractionpcsx2
2162a72831 GS: Bump GS Dump version and add transfer parameters to dump 2025-06-02 00:40:03 +02:00
refractionpcsx2
313666f85b GS: Store entire GS transfer state at TRXDIR write 2025-06-02 00:40:03 +02:00
TheTechnician27
e9d79263b4 UI: Fix Discord Rich Presence not activating in FSUI 2025-06-01 23:21:39 +02:00
JordanTheToaster
4ede6d65fd GameDB: Fix half right issue with Xtreme Bowling 2025-06-01 13:28:25 -04:00
PCSX2 Bot
14d2eee371 [ci skip] Qt: Update Base Translation. 2025-05-31 20:46:33 -04:00
chaoticgd
717f370be0 Debugger: Improve DockTabBar ownership workaround 2025-05-31 03:30:13 +02:00
refractionpcsx2
d05e4b9727 GS/HW: Adjust SpriteNoGaps check for vertical strips 2025-05-31 03:25:47 +02:00
TheTechnician27
697c53b4d8 UI: Standardize order of option groups 2025-05-31 02:51:20 +02:00
refractionpcsx2
d9b58ec3ce GS/HW: Invalidate single columns on small writes when formats mismatch 2025-05-31 02:49:35 +02:00
refractionpcsx2
6c8b37d7ca GS: Add column sizing to psm format information 2025-05-31 02:49:35 +02:00
refractionpcsx2
39f43e766d GameDB: Add Tex in RT to Sand Grain Studios games 2025-05-31 02:37:16 +02:00
refractionpcsx2
5379a13944 GS/HW: Read back 16bit target if read as 8H 2025-05-31 02:37:16 +02:00
chaoticgd
cd0c1607ef GameList: Clip flag and compatibility pixmaps to the available space 2025-05-31 02:35:34 +02:00
chaoticgd
26a4f71385 GameList: Prevent "Invalid" entry type appearing in filter list 2025-05-31 02:35:34 +02:00
chaoticgd
06b3e6ad71 GameList: Fix region flag icons for non-English languages 2025-05-31 02:35:34 +02:00
lightningterror
79d22a8d77 GS/HW: Add/adjust logs for failed texture creation. 2025-05-31 02:04:12 +02:00
lightningterror
9914212600 GS/GL: Add another hazard check, colclip hw.
Bonus, add missing texture copies info when doing shader copies.
2025-05-31 02:04:12 +02:00
lightningterror
765f55e67b GS/TC: Fine tune Frame buffer conversion. 2025-05-31 01:55:40 +02:00
PCSX2 Bot
2d922cc035 [ci skip] Qt: Update Base Translation. 2025-05-30 15:57:49 +02:00
TellowKrinkle
42e0625ab3 Interpreter: Fix FTOI on negative numbers 2025-05-29 13:15:09 +02:00
TellowKrinkle
c72e894fc7 iR5900: Faster FTOI 2025-05-29 13:15:09 +02:00
JordanTheToaster
5bc2342d47 MemoryCardFile: Fix memory card sorting on Linux 2025-05-29 13:07:27 +02:00
Ty
ea2b0b5e59 CI: Fix a regression for flathub uploads 2025-05-28 19:22:47 -04:00
refractionpcsx2
d70cc0221a GS/DX12: Fix HDR copy scissor area 2025-05-28 22:32:12 +02:00
refractionpcsx2
b12587b44e GameDB: Add Tex in RT for Bard's Tale 2025-05-28 18:33:27 +01:00
refractionpcsx2
c4708bdc35 GameDB: Add Tex in RT to required games 2025-05-28 18:01:49 +02:00
refractionpcsx2
1fa2c0bf50 GS/TC: Replace half right with Tex in RT, only update needed dirty 2025-05-28 18:01:49 +02:00
lightningterror
b4c70d357a GS/HW: Restore old coverage after updating mip layers. 2025-05-28 18:00:45 +02:00
PCSX2 Bot
f18262ee96 [ci skip] PAD: Update to latest controller database. 2025-05-26 21:18:58 +02:00
refractionpcsx2
c1f1761482 GS/HW: Invalidate cleared area if overlapping existing dirty 2025-05-26 13:45:47 +02:00
PCSX2 Bot
067c3eea16 [ci skip] Qt: Update Base Translation. 2025-05-26 02:03:40 +02:00
KamFretoZ
6957cc7001 FSUI: Fix save state duplicate entry 2025-05-26 01:46:30 +02:00
PCSX2 Bot
7dea23eea8 [ci skip] Qt: Update Base Translation. 2025-05-24 20:10:08 -04:00
TheTechnician27
319ec1f774 OSD: Fix performance overlay overwriting dump stats when shifted left 2025-05-24 12:33:32 +02:00
JohnSmith774
2079532e83 GameDB: Add memcard filters for some NTSC-J titles. (#12708)
Add memcard filters for OutRun2 SP - SPECIAL TOURS.
Add memcard filters for Another Century's Episode 2 Special Vocal Version.
Add memcard filters for Armored Core - Last Raven.
2025-05-24 12:32:03 +02:00
Mrlinkwii
de1d646fe9 github-workflows: Fix a broken link. 2025-05-24 12:30:08 +02:00
JordanTheToaster
4bc3ab6285 OSD: Add VSync to the OSD 2025-05-24 05:04:39 +02:00
JordanTheToaster
1adc9cbb49 GameDB: Various Fixes Part 4 (one with cheese) 2025-05-24 05:04:39 +02:00
refractionpcsx2
7cdf5eefc9 GS/TC: On clear delete overlapping depth targets 2025-05-24 05:03:48 +02:00
refractionpcsx2
c7ee72647d GS/HW: Improve texture shuffle/copy detection 2025-05-23 20:33:30 +02:00
Imre Eilertsen
5434af348e GameDB: Add listings for PSX Update Disc. 2025-05-23 13:31:40 +02:00
GovanifY
169cea0f01 docs: update in order to redirect from the GH wiki to our website 2025-05-22 17:43:42 +02:00
lightningterror
a8f3ec8184 GSClut: Get rid of m_read.dirty assert in GetAlphaMinMax32.
Replace it with a log.
2025-05-21 03:40:22 +02:00
lightningterror
7796b2a12b GS/HW: Add sanity/hazard checks for DATE and Texture barriers.
When checking for full barrier, also check if texture barriers
are supported.

Get rid of preprocessSel in DX11, turn off full barrier and related
stuff when not supported in rendererhw instead.

Check if StencilDate types are actually enabled, don't want to turn
it on for any others.
2025-05-21 03:40:22 +02:00
RedDevilus
d9cffd58ba GameDB: Add missing variants + fixes
Some games are missing some gamefixes or other type of fixes. Also added comment for like Energy Airforce where a legacy code never had a comment and removed mtvu for some variant + need to check in future to amend properly. Also missing entry SLPS-25875:
  name: "xxxHolic - Shigatsu Tsuitachi no Izayoi Sowa [Best Collection]"
2025-05-21 00:56:32 +02:00
refractionpcsx2
e566386240 GS/TC: only kill old misaligned targets on preload from previous frames. 2025-05-20 20:21:05 +02:00
refractionpcsx2
45af1e172f GS/HW: Improve shuffle pre-detection 2025-05-20 20:21:05 +02:00
refractionpcsx2
768b2e52a6 GS/TC: Don't allow Tex in RT 32bit target use as 16bit if not a shuffle 2025-05-20 20:21:05 +02:00
Ziemas
7abd2009b0 debugger: fix thread view row lookup 2025-05-20 11:57:11 -04:00
chaoticgd
46f075e891 DebugTools: Fix uninitialized variable in breakpoint code 2025-05-19 22:17:54 -04:00
lightningterror
832c381ac4 GS/HW: Allow partial depth copy on dx12. 2025-05-19 22:10:42 +02:00
SquishyLeaf
e021282264 CI: Fix script building universal dependencies on macOS
- Skip arm64 binaries when looking for x86_64 binaries to merge

- Change Qt download link to archive

- Build universal binaries for libjpegturbo, PlutoVG and PlutoSVG
2025-05-19 14:21:14 -04:00
PCSX2 Bot
2a60d385c6 [ci skip] PAD: Update to latest controller database. 2025-05-19 18:42:36 +02:00
refractionpcsx2
4d37e35675 GS/TC: Delete dirty rt's in src lookup + usert in rt on 3 draw old rt's 2025-05-18 11:13:35 +02:00
refractionpcsx2
df3868a280 GS/HW: Avoid target height mistakes on shuffles + Update new src == rt 2025-05-18 11:13:35 +02:00
lightningterror
0799bb8cf1 GS/DX: DX requires a copy to sample the depth buffer. 2025-05-17 22:54:04 +02:00
lightningterror
69048dede4 GS/DX11: Merge CloneTexture with CopyRect.
Unified between renderers, easier to make shared changes.
2025-05-17 22:54:04 +02:00
lightningterror
76df6d1f43 GS/GL: Check for texture creation hazard for fb copy. 2025-05-17 22:54:04 +02:00
refractionpcsx2
2d03b21f2b Formatting: Clean up some if spaces 2025-05-17 22:47:38 +02:00
refractionpcsx2
62cbd44933 GS/HW: Fix up offset Z behaviour + work even when RT isn't offset. 2025-05-17 22:47:38 +02:00
refractionpcsx2
44c8f6d8b0 GS/HW: Correct valid area for depth when taking alpha from rt. Add Tex in RT to Area 51 2025-05-17 22:47:38 +02:00
refractionpcsx2
f3fc1dd59c GS/HW: Fix some regressions relating to overlapping targets and valid rect + rgb 2025-05-17 22:47:38 +02:00
lightningterror
708931e48b GS/HW: Clean up HandleTextureHazards.
Warnings, initializations, dereferencing null pointers.
2025-05-17 22:46:37 +02:00
TheLastRar
81800d2883 CMake: Support more package configurations to import with a Devel build 2025-05-17 09:39:56 -04:00
TellowKrinkle
d0411d7ddf Core:macOS: Initialize all address variables passed to vm_map 2025-05-16 12:41:09 +02:00
Ty
70e232cab3 GitHub: Make authors disclose whether they used AI or not.
Using checkboxes implicitly made it a task list that cluttered up the PR list.

[ci skip]
2025-05-16 12:27:15 +02:00
refractionpcsx2
f90396bda4 GS/DX12: Fix command list not flushing when in surfaceless mode. 2025-05-16 12:25:16 +02:00
lightningterror
ae8808b86e GS/HW: Backport some tex is fb shaders to dx and opengl. 2025-05-15 20:08:16 +02:00
lightningterror
8d5b827432 GS/HW: Fix texture copies when tex is fb draw.
Fixes an issue with texture copies didn't work properly on tex is fb draw:
Fixes Hitman Blood Money on minimum blend.

DX can't do partial depth copy so do a shader based copy which works.
Fixes a bunch of games that couldn't do partial depth copy on dx.
2025-05-15 20:08:16 +02:00
lightningterror
cb672697e7 FullscreenUI: Fix -Wlogical-op-parentheses warning. 2025-05-15 20:08:16 +02:00
JordanTheToaster
0bae9fc29b Deps: Update SDL3 to 3.2.14 2025-05-15 18:43:45 +02:00
lightningterror
49c8b68700 GS/HW: Enable feedback loop if channel shuffle enabled barriers. 2025-05-15 18:43:35 +02:00
refractionpcsx2
1e2fcd17e0 GS/HW: Fix new target creation width when doing a page copy 2025-05-15 01:45:11 +02:00
refractionpcsx2
468b9d2655 GS/TC: Use frame width on PSMT8H read if target width doesn't match frame on Tex in RT.
Also changed LookupSource/Depth to receive the whole FRAME struct, not just the pointer.
2025-05-15 01:45:11 +02:00
refractionpcsx2
a3305ff791 GS/TC: Used unwrapped end blocks for combine 2025-05-15 01:45:11 +02:00
PCSX2 Bot
814b0d5873 [ci skip] Qt: Update Base Translation. 2025-05-13 20:00:55 +02:00
refractionpcsx2
863f3e82ac GS/HW: Combine target using the drawn area, not valid area (some of this can be garbage) 2025-05-13 19:56:39 +02:00
Ty
529c756458 GitHub: Require PR authors to disclose AI usage
[noci]
2025-05-12 18:12:02 -04:00
TheLastRar
67b98dbdaa Deps: Also build KDDockWidgets as Debug 2025-05-12 22:35:07 +02:00
TheLastRar
a16981cbe5 CMake: Pick release packages when building with devel 2025-05-12 22:35:07 +02:00
lightningterror
777bf338fa GS/HW: Fix tex is fb log for palette conversion. 2025-05-12 22:23:14 +02:00
lightningterror
3ba8d9ebff GS/HW: Move texture sampling before EmulateBlending.
Tex is fb setting is done in Texture sampler which resulted
in hw blend being applied before with barriers on.

Tex is fb can use sw blend so this optimizes that with no cost.

Also fixes an issue where hw blend was on with fbfetch.
2025-05-12 21:40:51 +02:00
lightningterror
db5384e19c GS/HW: Add hazard check for date depth buffer creation.
Add fallback to PrimIDTracking if possible.
2025-05-12 21:39:43 +02:00
lightningterror
fd3984287b GS/HW: Add hazard check for temporary depth buffer creation. 2025-05-12 21:39:43 +02:00
lightningterror
31dbea4e0b GS: Retry texture creation after initial retry fails.
This should've been done.
2025-05-12 21:39:43 +02:00
refractionpcsx2
0992e784f1 GS/HW: Tighten shuffle check on source lookup 2025-05-12 21:28:41 +02:00
refractionpcsx2
c59cc1fbb7 GS/TC: Kill partially dirty RT in RT targets which haven't been used recently 2025-05-12 21:28:41 +02:00
refractionpcsx2
903edc79f7 GS/TC: Alter RT in RT target lookup further and expand target on Move if needed 2025-05-12 21:28:41 +02:00
refractionpcsx2
4c94c69f02 GS: Clear Adaptive Interlacing buffer when outputs are disabled 2025-05-12 21:28:41 +02:00
refractionpcsx2
4c57c214bd GS/HW: Expand target to fix requested source rect when dirty covers area 2025-05-12 21:28:41 +02:00
refractionpcsx2
1cbeea3016 GS/HW: Fix up inside target lookup behaviour in HW renderer 2025-05-12 21:28:41 +02:00
PCSX2 Bot
0d18738192 [ci skip] PAD: Update to latest controller database. 2025-05-12 21:01:31 +02:00
JordanTheToaster
1e7c7db29b GameDB: Various Fixes Part 3 Because I Forgot More 2025-05-12 01:05:14 +02:00
PCSX2 Bot
7c098a2665 [ci skip] Qt: Update Base Translation. 2025-05-11 02:02:52 +02:00
Ty
3ab539be1e CDVD: Fix for non-PCH builds 2025-05-10 15:38:02 -04:00
Ty
b46d5e4efc CI: Disable PCH on Linux 2025-05-10 15:38:02 -04:00
Jordan
db06e11a7c GameDB: RT in RT Part Deux.
Adds fixes such as ATNWTO to around 104 games and adjusts the names of even more. Don't ask me what I fixed I don't even remember at this point.
2025-05-10 19:54:21 +02:00
KamFretoZ
757ed33e4f FSUI: Add ImDrawList helper for drawing SVG images
Co-Authored-By: TheLastRar <TheLastRar@users.noreply.github.com>
2025-05-10 19:53:04 +02:00
KamFretoZ
8eb5dfce85 Qt/FSUI: Update branding to the about section 2025-05-10 19:53:04 +02:00
PCSX2 Bot
f34db72a97 [ci skip] Qt: Update Base Translation. 2025-05-09 18:13:36 -04:00
chaoticgd
3dfdc8ee6f Debugger: Fix a typo in the memory search view 2025-05-08 22:16:38 +02:00
chaoticgd
5a846296ca Docs: Add even more missing third party licenses 2025-05-08 09:23:22 -04:00
lightningterror
0bea015dcd GS/HW: Adjust blending when tex is fb.
Prefer sw blend when it's a channel shuffle with barriers enabled.
Prefer sw blend if it's tex is fb when there is no overlap, fb is already
being read so we can use sw blend.
Swap full barriers with one barrier if it's a channel shuffle,
prims shouldn't overlap anyway.
2025-05-08 05:05:49 +02:00
Ziemas
fe0b401edd vtlb: silence backpatch log spam
Since this works well right now I don't think we need this.
2025-05-07 19:46:33 +02:00
Ziemas
69f8e82c3f gs: Missing cmath include 2025-05-07 19:31:44 +02:00
chaoticgd
79d3d43a85 Docs: Update Zydis license 2025-05-07 14:40:51 +02:00
chaoticgd
e8a7f596bc Docs: Update LZ4 license 2025-05-07 14:40:51 +02:00
chaoticgd
a096c00b68 Docs: Escape third party license text with HTML entities 2025-05-07 14:40:51 +02:00
chaoticgd
e284ef8906 Docs: Add index section to third party licenses file 2025-05-07 14:40:51 +02:00
chaoticgd
154a8750dc Docs: Add more missing third party licenses 2025-05-07 14:40:51 +02:00
chaoticgd
0de4ca4a19 Docs: Sort third party licenses alphabetically 2025-05-07 14:40:51 +02:00
lightningterror
23c9c9f72e GS/TC: Also update tc logs with prefixes. 2025-05-07 14:39:12 +02:00
lightningterror
96ff10c692 GS/HW: Update logs.
Cleanup/remove some logs, name them properly with HW prefix.
2025-05-07 14:39:12 +02:00
lightningterror
30bc7ea132 GSDumpRunner: Fix some userhack arguments not working. 2025-05-07 13:39:12 +02:00
lightningterror
d9ea1591b0 GS/HW: Enable tex-is-fb on channel shuffles when barriers aren't supported. 2025-05-07 02:50:24 +02:00
PCSX2 Bot
0cc1bb5ad8 [ci skip] Qt: Update Base Translation. 2025-05-07 02:08:22 +02:00
TheLastRar
509bbd7605 Qt: Allow translating the default adapter text
Strangely, the needed logic was already present (but partly unused)
Also adjust loading logic to match 1b50057764
2025-05-05 22:12:15 -04:00
PCSX2 Bot
4632afe1ab [ci skip] PAD: Update to latest controller database. 2025-05-05 19:25:52 +02:00
JordanTheToaster
6680698ba6 Deps: Update SDL to 3.2.12 2025-05-05 19:25:41 +02:00
JordanTheToaster
2b06b12ca2 3rdparty: Update googletest to v1.16.0 2025-05-05 19:25:41 +02:00
JordanTheToaster
9b53916e06 Deps: Update harfbuzz to 11.2.0 2025-05-05 19:25:41 +02:00
JordanTheToaster
f3ce1dd7a3 Deps: Update libpng to 1.6.48
a
2025-05-05 19:25:41 +02:00
JordanTheToaster
91b0426d68 Deps: Update LZ4 to v1.10.0 2025-05-05 19:25:41 +02:00
TheTechnician27
63a5b95939 Audio: Fix global settings overriding per-game ones 2025-05-04 21:55:22 +02:00
Gilad
480a3fe171 GameDB: Add HPO:ATN to Fatal Frame III 2025-05-04 21:32:41 +02:00
Gilad
b160eac49f GameDB: Add ATNTWO to Fatal Frame III 2025-05-04 21:32:41 +02:00
ElTioRata
37ba82b8b7 GameDB: HPO Native w/TO for Resident Evil Dead Aim
Fixes water lines on sewer levels.
2025-05-04 21:31:10 +02:00
chaoticgd
2d604145f1 Docs: Remove advertising gunk from GPL license file 2025-05-04 14:55:45 +02:00
chaoticgd
d60a6df313 Docs: Improve formatting of third party licenses 2025-05-04 14:55:45 +02:00
chaoticgd
236d9e3028 Docs: Add missing third party licenses 2025-05-04 14:55:45 +02:00
chaoticgd
175327e711 Docs: Update LGPL remnant 2025-05-04 14:55:45 +02:00
lightningterror
6342f99504 GS/HW: Fix copy range for shuffles.
We should be using the sizes based on source instead of target
when clamping depth range.
2025-05-04 09:07:45 +02:00
PCSX2 Bot
6164ae9f60 [ci skip] Qt: Update Base Translation. 2025-05-03 20:08:13 -04:00
516 changed files with 254840 additions and 175678 deletions

View File

@@ -1,54 +0,0 @@
# So you want to contribute to PCSX2? Great
As a first step, please review these links as they'll help you understand how the development of PCSX2 works.
* [Just Starting Out](#just-starting-out)
* [Issue Reporting](#issue-reporting)
* [Pull Request Guidelines](#pull-request-guidelines)
* [General Documentation And Coding Strategies](#general-documentation-and-coding-strategies)
* [Tasks](#tasks)
## Just Starting Out
* If you're unfamilar with git, check out this [brief introduction to Git](https://github.com/PCSX2/pcsx2/wiki/07-Git-survival-guide)
* [How to build PCSX2 for Windows](https://github.com/PCSX2/pcsx2/wiki/12-Building-on-Windows)
* [How to build PCSX2 for Linux](https://github.com/PCSX2/pcsx2/wiki/10-Building-on-Linux)
## Issue Reporting
* [How to write a useful issue](https://github.com/PCSX2/pcsx2/wiki/How-to-create-useful-and-valid-issues)
## Pull Request Guidelines
The following is a list of *general* style recommendations that will make reviewing and merging easier:
* Commit Messages
* Please try to prefix your commit message, indicating what area of the project was modified.
* For example `gs: message...`.
* Looking at the project's commit history will help with keeping prefixes consistent overtime, *there is no strictly enforced list*.
* Try to keep messages brief and informative
* Remove unnecessary commits and squash commits together when appropriate.
* If you are not familiar with rebasing with git, check out the following resources:
* CLI - https://thoughtbot.com/blog/git-interactive-rebase-squash-amend-rewriting-history
* GUI (SourceTree) - https://www.atlassian.com/blog/sourcetree/interactive-rebase-sourcetree
* Code Styling and Formatting
* [Consult the style guide](https://github.com/tadanokojin/pcsx2/blob/coding-guide/pcsx2/Docs/Coding_Guidelines.md)
* Run `clang-format` using the configuration file in the root of the repository
* Visual Studio Setup - https://devblogs.microsoft.com/cppblog/clangformat-support-in-visual-studio-2017-15-7-preview-1/
* IMPORTANT - if you are running `clang-format` on unrelated changes (ie. formatting an entire file), please do so in a separate commit.
* If you cannot scope your `clang-format` to just your changes and do not want to format unrelated code. Try your best to stick with the existing formatting already established in the file in question.
## General Documentation And Coding Strategies
* [Commenting Etiquette](https://github.com/PCSX2/pcsx2/wiki/06-Commenting-Etiquette)
* [Coding style](https://github.com/PCSX2/pcsx2/wiki/Code-Formatting-Guidelines)
* [More comprehensive style-guide (Currently in Draft)](https://github.com/tadanokojin/pcsx2/blob/coding-guide/pcsx2/Docs/Coding_Guidelines.md)
## Tasks
* [Issues](https://github.com/PCSX2/pcsx2/issues)

View File

@@ -70,6 +70,8 @@ body:
- Windows 11
- Windows 10 (64bit)
- Linux (64bit) - Specify distro below
- macOS 26 (Tahoe)
- macOS 15 (Sequoia)
- macOS 14 (Sonoma)
- macOS 13 (Ventura)
- macOS 12 (Monterey)

View File

@@ -66,6 +66,10 @@ body:
Performance issues as a result of not meeting our hardware requirements are not valid.
Please read our troubleshooting pages and our issue reporting guide.
- [Troubleshooting page](https://pcsx2.net/docs/category/troubleshooting).
- [Issue reporting guide](https://pcsx2.net/docs/troubleshooting/identify).
We are **not** accepting issues related to the **libretro** core. The libretro core is being maintained separately at this time
- type: input
id: rev
@@ -83,6 +87,8 @@ body:
- Windows 11
- Windows 10 (64bit)
- Linux (64bit) - Specify distro below
- macOS 26 (Tahoe)
- macOS 15 (Sequoia)
- macOS 14 (Sonoma)
- macOS 13 (Ventura)
- macOS 12 (Monterey)
@@ -100,14 +106,14 @@ body:
id: cpu
attributes:
label: CPU
placeholder: "Example: i5-7600"
placeholder: "Example: Intel i5 12400F"
validations:
required: true
- type: input
id: gpu
attributes:
label: GPU
placeholder: "Example: GTX 1070"
placeholder: "Example: Nvidia RTX 4060"
validations:
required: true
- type: textarea

View File

@@ -6,3 +6,6 @@
### Suggested Testing Steps
<!-- If applicable, including examples you've already tested with / recommendations for how to test further is very helpful! -->
### Did you use AI to help find, test, or implement this issue or feature?
<!-- Answer yes or no. If you answer yes, please provide a brief explanation how. -->

View File

@@ -6,36 +6,45 @@ on:
workflow_dispatch: # As well as manually.
jobs:
check:
if: github.repository == 'PCSX2/pcsx2'
name: "Check if release is needed"
runs-on: ubuntu-latest
timeout-minutes: 180
outputs:
PCSX2_RELEASE: ${{ steps.getinfo.outputs.PCSX2_RELEASE }}
FLATHUB_RELEASE: ${{ steps.getinfo.outputs.FLATHUB_RELEASE }}
steps:
- name: Get latest tag and Flathub release
id: getinfo
env:
GH_TOKEN: ${{ github.token }}
run: |
PCSX2_RELEASE=$(gh api -H 'Accept: application/vnd.github+json' -H 'X-GitHub-Api-Version: 2022-11-28' /repos/PCSX2/pcsx2/releases | jq -r '.[0].tag_name')
FLATHUB_RELEASE=$(curl -L -s https://flathub.org/api/v2/appstream/net.pcsx2.PCSX2 | jq -r '.releases | max_by(.version) | .version')
echo "Latest PCSX2 release is: '${PCSX2_RELEASE}'"
echo "Latest Flathub release is: '${FLATHUB_RELEASE}'"
PCSX2_RELEASE=$(echo $PCSX2_RELEASE | sed 's/[^0-9]*//g')
FLATHUB_RELEASE=$(echo $FLATHUB_RELEASE | sed 's/[^0-9]*//g')
echo "PCSX2_RELEASE=${PCSX2_RELEASE}" >> "$GITHUB_OUTPUT"
echo "FLATHUB_RELEASE=${FLATHUB_RELEASE}" >> "$GITHUB_OUTPUT"
# check is disabled as the flathub api does not give us beta repository information
# Alternatively we can "flatpak remote-info or parse the appstream directly for the beta repo"
# Maybe in the future if we don't want to publish the same version twice if we get no commits
# for 24 hours.
# check:
# if: github.repository == 'PCSX2/pcsx2'
# name: "Check if release is needed"
# runs-on: ubuntu-latest
# timeout-minutes: 180
# outputs:
# PCSX2_RELEASE: ${{ steps.getinfo.outputs.PCSX2_RELEASE }}
# FLATHUB_RELEASE: ${{ steps.getinfo.outputs.FLATHUB_RELEASE }}
# steps:
# - name: Get latest tag and Flathub release
# id: getinfo
# env:
# GH_TOKEN: ${{ github.token }}
# run: |
# PCSX2_RELEASE=$(gh api -H 'Accept: application/vnd.github+json' -H 'X-GitHub-Api-Version: 2022-11-28' /repos/PCSX2/pcsx2/releases | jq -r '.[0].tag_name')
# FLATHUB_RELEASE=$(curl -L -s https://flathub.org/api/v2/appstream/net.pcsx2.PCSX2 | jq -r '.releases | max_by(.version) | .version')
# echo "Latest PCSX2 release is: '${PCSX2_RELEASE}'"
# echo "Latest Flathub release is: '${FLATHUB_RELEASE}'"
# PCSX2_RELEASE=$(echo $PCSX2_RELEASE | sed 's/[^0-9]*//g')
# FLATHUB_RELEASE=$(echo $FLATHUB_RELEASE | sed 's/[^0-9]*//g')
# echo "PCSX2_RELEASE=${PCSX2_RELEASE}" >> "$GITHUB_OUTPUT"
# echo "FLATHUB_RELEASE=${FLATHUB_RELEASE}" >> "$GITHUB_OUTPUT"
build:
needs: check
# needs: check
# outputs are automatically compared as strings. This doesn't work in our favour
# Use fromJson() to convert them to proper integers...
# see: https://github.com/github/docs/pull/25870
# and: https://github.com/orgs/community/discussions/57480
#if: fromJson(needs.check.outputs.FLATHUB_RELEASE) < fromJson(needs.check.outputs.PCSX2_RELEASE)
# if: fromJson(needs.check.outputs.FLATHUB_RELEASE) < fromJson(needs.check.outputs.PCSX2_RELEASE)
# As the check step is disabled, perform repository check here
if: github.repository == 'PCSX2/pcsx2'
name: "Build and publish Flatpak"
uses: ./.github/workflows/linux_build_flatpak.yml
with:

View File

@@ -54,19 +54,16 @@ jobs:
uses: actions/checkout@v4
with:
set-safe-directory: ${{ env.GITHUB_WORKSPACE }}
# 10 here, since the odds of having 10 untagged commits in a row should be slim to none
# This is required for the tagging logic in generate-metainfo.sh
fetch-depth: 10
fetch-tags: true
# Work around container ownership issue
- name: Set Safe Directory
shell: bash
run: git config --global --add safe.directory "*"
# Hackity hack. When running the workflow on a schedule, we don't have the tag,
# it doesn't fetch tags, therefore we don't get a version. So grab them manually.
# actions/checkout elides tags, fetch them primarily for releases
- name: Fetch tags
if: ${{ inputs.fetchTags }}
run: git fetch --tags --no-recurse-submodules
- name: Add stable release identifier file
if: ${{ inputs.stableBuild == true || inputs.stableBuild == 'true' }}
shell: bash
@@ -125,14 +122,14 @@ jobs:
cache: true
restore-cache: true
cache-key: ${{ inputs.os }} ${{ inputs.platform }} ${{ inputs.compiler }} flatpak ${{ hashFiles('.github/workflows/scripts/linux/flatpak/**/*.json') }}
#- name: Validate build
# run: |
# flatpak-builder-lint repo repo
- name: Push to Flathub (beta)
if: ${{ inputs.publish == true && (inputs.stableBuild == false || inputs.stableBuild == 'false') }}
uses: flatpak/flatpak-github-actions/flatpak-builder@10a3c29f0162516f0f68006be14c92f34bd4fa6c
uses: flatpak/flatpak-github-actions/flat-manager@10a3c29f0162516f0f68006be14c92f34bd4fa6c
with:
flat-manager-url: https://hub.flathub.org/
repository: beta
@@ -141,7 +138,7 @@ jobs:
- name: Push to Flathub (stable)
if: ${{ inputs.publish == true && (inputs.stableBuild == true || inputs.stableBuild == 'true') }}
uses: flatpak/flatpak-github-actions/flatpak-builder@10a3c29f0162516f0f68006be14c92f34bd4fa6c
uses: flatpak/flatpak-github-actions/flat-manager@10a3c29f0162516f0f68006be14c92f34bd4fa6c
with:
flat-manager-url: https://hub.flathub.org/
repository: stable

View File

@@ -140,9 +140,12 @@ jobs:
-DCMAKE_CXX_COMPILER=clang++-17 \
-DCMAKE_EXE_LINKER_FLAGS_INIT="-fuse-ld=lld" \
-DCMAKE_MODULE_LINKER_FLAGS_INIT="-fuse-ld=lld" \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
-DENABLE_SETCAP=OFF \
-DDISABLE_ADVANCE_SIMD=TRUE \
-DUSE_LINKED_FFMPEG=ON \
-DCMAKE_DISABLE_PRECOMPILE_HEADERS=ON \
$ADDITIONAL_CMAKE_ARGS
- name: Build PCSX2

View File

@@ -68,7 +68,7 @@ jobs:
mv ./release-notes.md ${GITHUB_WORKSPACE}/release-notes.md
- name: Create a GitHub Release (Manual)
uses: softprops/action-gh-release@da05d552573ad5aba039eaac05058a918a7bf631
uses: softprops/action-gh-release@72f2c25fcb47643c292f7107632f7a47c1df5cd8
if: steps.tag_version.outputs.new_tag && github.event_name == 'workflow_dispatch'
with:
body_path: ./release-notes.md
@@ -77,7 +77,7 @@ jobs:
tag_name: ${{ steps.tag_version.outputs.new_tag }}
- name: Create a GitHub Release (Push)
uses: softprops/action-gh-release@da05d552573ad5aba039eaac05058a918a7bf631
uses: softprops/action-gh-release@72f2c25fcb47643c292f7107632f7a47c1df5cd8
if: steps.tag_version.outputs.new_tag && github.event_name != 'workflow_dispatch'
with:
body_path: ./release-notes.md

View File

@@ -45,16 +45,6 @@ declare -a MANUAL_LIBS=(
"libshaderc_shared.so.1"
)
declare -a MANUAL_QT_LIBS=(
"libQt6WaylandEglClientHwIntegration.so.6"
)
declare -a MANUAL_QT_PLUGINS=(
"wayland-decoration-client"
"wayland-graphics-integration-client"
"wayland-shell-integration"
)
declare -a REMOVE_LIBS=(
'libwayland-client.so*'
'libwayland-cursor.so*'
@@ -66,7 +56,6 @@ set -e
LINUXDEPLOY=./linuxdeploy-x86_64.AppImage
LINUXDEPLOY_PLUGIN_QT=./linuxdeploy-plugin-qt-x86_64.AppImage
APPIMAGETOOL=./appimagetool-x86_64.AppImage
PATCHELF=patchelf
if [ ! -f "$LINUXDEPLOY" ]; then
"$PCSX2DIR/tools/retry.sh" wget -O "$LINUXDEPLOY" https://github.com/linuxdeploy/linuxdeploy/releases/download/continuous/linuxdeploy-x86_64.AppImage
@@ -125,7 +114,10 @@ cp "$PCSX2DIR/.github/workflows/scripts/linux/pcsx2-qt.desktop" "net.pcsx2.PCSX2
cp "$PCSX2DIR/bin/resources/icons/AppIconLarge.png" "PCSX2.png"
echo "Running linuxdeploy to create AppDir..."
EXTRA_QT_PLUGINS="core;gui;svg;waylandclient;widgets;xcbqpa" \
# The wayland platform plugin requires the plugins deployed for the waylandcompositor module
# Interestingly, specifying the module doesn't copy the module, only the required plugins for it
# https://github.com/linuxdeploy/linuxdeploy-plugin-qt/issues/160#issuecomment-2655543893
EXTRA_QT_MODULES="core;gui;svg;waylandclient;waylandcompositor;widgets;xcbqpa" \
EXTRA_PLATFORM_PLUGINS="libqwayland-egl.so;libqwayland-generic.so" \
DEPLOY_PLATFORM_THEMES="1" \
QMAKE="$DEPSDIR/bin/qmake" \
@@ -136,34 +128,6 @@ $LINUXDEPLOY --plugin qt --appdir="$OUTDIR" --executable="$BUILDDIR/bin/pcsx2-qt
echo "Copying resources into AppDir..."
cp -a "$BUILDDIR/bin/resources" "$OUTDIR/usr/bin"
# LinuxDeploy's Qt plugin doesn't include Wayland support. So manually copy in the additional Wayland libraries.
echo "Copying Qt Wayland libraries..."
for lib in "${MANUAL_QT_LIBS[@]}"; do
srcpath="$DEPSDIR/lib/$lib"
dstpath="$OUTDIR/usr/lib/$lib"
echo " $srcpath -> $dstpath"
cp "$srcpath" "$dstpath"
$PATCHELF --set-rpath '$ORIGIN' "$dstpath"
done
# .. and plugins.
echo "Copying Qt Wayland plugins..."
for GROUP in "${MANUAL_QT_PLUGINS[@]}"; do
srcpath="$DEPSDIR/plugins/$GROUP"
dstpath="$OUTDIR/usr/plugins/$GROUP"
echo " $srcpath -> $dstpath"
mkdir -p "$dstpath"
for srcsopath in $(find "$DEPSDIR/plugins/$GROUP" -iname '*.so'); do
# This is ../../ because it's usually plugins/group/name.so
soname=$(basename "$srcsopath")
dstsopath="$dstpath/$soname"
echo " $srcsopath -> $dstsopath"
cp "$srcsopath" "$dstsopath"
$PATCHELF --set-rpath '$ORIGIN/../../lib:$ORIGIN' "$dstsopath"
done
done
# Why do we have to manually remove these libs? Because the linuxdeploy Qt plugin
# copies them, not the "main" linuxdeploy binary, and plugins don't inherit the
# include list...

View File

@@ -16,20 +16,20 @@ fi
LIBBACKTRACE=ad106d5fdd5d960bd33fae1c48a351af567fd075
LIBJPEGTURBO=3.1.0
LIBPNG=1.6.45
LIBPNG=1.6.48
LIBWEBP=1.5.0
LZ4=b8fd2d15309dd4e605070bd4486e26b6ef814e29
SDL=SDL3-3.2.10
QT=6.9.0
SDL=SDL3-3.2.16
QT=6.9.1
LZ4=1.10.0
ZSTD=1.5.7
KDDOCKWIDGETS=2.2.3
PLUTOVG=0.0.13
PLUTOSVG=0.0.6
PLUTOVG=1.1.0
PLUTOSVG=0.0.7
SHADERC=2024.1
SHADERC_GLSLANG=142052fa30f9eca191aa9dcf65359fcaed09eeec
SHADERC_SPIRVHEADERS=5e3ad389ee56fca27c9705d093ae5387ce404df4
SHADERC_SPIRVTOOLS=dd4b663e13c07fea4fbb3f70c1c91c86731099f7
SHADERC=2025.3
SHADERC_GLSLANG=efd24d75bcbc55620e759f6bf42c45a32abac5f8
SHADERC_SPIRVHEADERS=2a611a970fdbc41ac2e3e328802aed9985352dca
SHADERC_SPIRVTOOLS=33e02568181e3312f49a3cf33df470bf96ef293a
mkdir -p deps-build
cd deps-build
@@ -37,24 +37,24 @@ cd deps-build
cat > SHASUMS <<EOF
fd6f417fe9e3a071cf1424a5152d926a34c4a3c5070745470be6cf12a404ed79 $LIBBACKTRACE.zip
9564c72b1dfd1d6fe6274c5f95a8d989b59854575d4bbee44ade7bc17aa9bc93 libjpeg-turbo-$LIBJPEGTURBO.tar.gz
926485350139ffb51ef69760db35f78846c805fef3d59bfdcb2fba704663f370 libpng-$LIBPNG.tar.xz
46fd06ff37db1db64c0dc288d78a3f5efd23ad9ac41561193f983e20937ece03 libpng-$LIBPNG.tar.xz
7d6fab70cf844bf6769077bd5d7a74893f8ffd4dfb42861745750c63c2a5c92c libwebp-$LIBWEBP.tar.gz
0728800155f3ed0a0c87e03addbd30ecbe374f7b080678bbca1506051d50dec3 $LZ4.tar.gz
f87be7b4dec66db4098e9c167b2aa34e2ca10aeb5443bdde95ae03185ed513e0 $SDL.tar.gz
6340e58879b2d15830c8460d2f589a385c444d1faa2a4828a9626c7322562be8 $SDL.tar.gz
537512904744b35e232912055ccf8ec66d768639ff3abe5788d90d792ec5f48b lz4-$LZ4.tar.gz
eb33e51f49a15e023950cd7825ca74a4a2b43db8354825ac24fc1b7ee09e6fa3 zstd-$ZSTD.tar.gz
c1800c2ea835801af04a05d4a32321d79a93954ee3ae2172bbeacf13d1f0598c qtbase-everywhere-src-$QT.tar.xz
2047c6242a57bf97cf40079fa9f91752c137cd9ae84760faa9a2e5e8a440606f qtimageformats-everywhere-src-$QT.tar.xz
ec359d930c95935ea48af58b100c2f5d0d275968ec8ca1e0e76629b7159215fc qtsvg-everywhere-src-$QT.tar.xz
fa645589cc3f939022401a926825972a44277dead8ec8607d9f2662e6529c9a4 qttools-everywhere-src-$QT.tar.xz
1d5581ef5fc7c7bc556f2403017983683993bbebfcdf977ef8f180f604668c3f qttranslations-everywhere-src-$QT.tar.xz
503416fcb04db503bd130e6a49c45e3e546f091e83406f774a0c703130c91805 qtwayland-everywhere-src-$QT.tar.xz
eb3b5f0c16313d34f208d90c2fa1e588a23283eed63b101edd5422be6165d528 shaderc-$SHADERC.tar.gz
aa27e4454ce631c5a17924ce0624eac736da19fc6f5a2ab15a6c58da7b36950f shaderc-glslang-$SHADERC_GLSLANG.tar.gz
5d866ce34a4b6908e262e5ebfffc0a5e11dd411640b5f24c85a80ad44c0d4697 shaderc-spirv-headers-$SHADERC_SPIRVHEADERS.tar.gz
03ee1a2c06f3b61008478f4abe9423454e53e580b9488b47c8071547c6a9db47 shaderc-spirv-tools-$SHADERC_SPIRVTOOLS.tar.gz
40caedbf83cc9a1959610830563565889878bc95f115868bbf545d1914acf28e qtbase-everywhere-src-$QT.tar.xz
ebe9f238daaf9bb752c7233edadf4af33fc4fa30d914936812b6410d3af1577c qtimageformats-everywhere-src-$QT.tar.xz
2dfc5de5fd891ff2afd9861e519bf1a26e6deb729b3133f68a28ba763c9abbd5 qtsvg-everywhere-src-$QT.tar.xz
90c4a562f4ccfd043fd99f34c600853e0b5ba9babc6ec616c0f306f2ce3f4b4c qttools-everywhere-src-$QT.tar.xz
9761a1a555f447cdeba79fdec6a705dee8a7882ac10c12e85f49467ddd01a741 qttranslations-everywhere-src-$QT.tar.xz
7d21ea0e687180ebb19b9a1f86ae9cfa7a25b4f02d5db05ec834164409932e3e qtwayland-everywhere-src-$QT.tar.xz
a8e4a25e5c2686fd36981e527ed05e451fcfc226bddf350f4e76181371190937 shaderc-$SHADERC.tar.gz
9427deccbdf4bde6a269938df38c6bd75247493786a310d8d733a2c82065ef47 shaderc-glslang-$SHADERC_GLSLANG.tar.gz
c2225a49c3d7efa5c4f4ce4a6b42081e6ea3daca376f3353d9d7c2722d77a28a shaderc-spirv-headers-$SHADERC_SPIRVHEADERS.tar.gz
44d1005880c583fc00a0fb41c839214c68214b000ea8dcb54d352732fee600ff shaderc-spirv-tools-$SHADERC_SPIRVTOOLS.tar.gz
b8529755b2d54205341766ae168e83177c6120660539f9afba71af6bca4b81ec KDDockWidgets-$KDDOCKWIDGETS.tar.gz
f49d62709d6bf1808ddc9b8f71e22a755484f75c7bbb0fb368f7fb2ffc7cf645 plutovg-$PLUTOVG.tar.gz
01f8aee511bd587a602a166642a96522cc9522efd1e38c2d00e4fbc0aa22d7a0 plutosvg-$PLUTOSVG.tar.gz
8aa9860519c407890668c29998e8bb88896ef6a2e6d7ce5ac1e57f18d79e1525 plutovg-$PLUTOVG.tar.gz
78561b571ac224030cdc450ca2986b4de915c2ba7616004a6d71a379bffd15f3 plutosvg-$PLUTOSVG.tar.gz
EOF
curl -L \
@@ -62,7 +62,7 @@ curl -L \
-O "https://github.com/libjpeg-turbo/libjpeg-turbo/releases/download/$LIBJPEGTURBO/libjpeg-turbo-$LIBJPEGTURBO.tar.gz" \
-O "https://downloads.sourceforge.net/project/libpng/libpng16/$LIBPNG/libpng-$LIBPNG.tar.xz" \
-O "https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-$LIBWEBP.tar.gz" \
-O "https://github.com/lz4/lz4/archive/$LZ4.tar.gz" \
-O "https://github.com/lz4/lz4/releases/download/v$LZ4/lz4-$LZ4.tar.gz" \
-O "https://libsdl.org/release/$SDL.tar.gz" \
-O "https://github.com/facebook/zstd/releases/download/v$ZSTD/zstd-$ZSTD.tar.gz" \
-O "https://download.qt.io/official_releases/qt/${QT%.*}/$QT/submodules/qtbase-everywhere-src-$QT.tar.xz" \
@@ -110,7 +110,7 @@ cd ..
echo "Building LZ4..."
rm -fr "lz4-$LZ4"
tar xf "$LZ4.tar.gz"
tar xf "lz4-$LZ4.tar.gz"
cd "lz4-$LZ4"
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH="$INSTALLDIR" -DCMAKE_INSTALL_PREFIX="$INSTALLDIR" -DBUILD_SHARED_LIBS=ON -DLZ4_BUILD_CLI=OFF -DLZ4_BUILD_LEGACY_LZ4C=OFF -B build-dir -G Ninja build/cmake
cmake --build build-dir --parallel

View File

@@ -14,8 +14,8 @@
"sources": [
{
"type": "archive",
"url": "https://libsdl.org/release/SDL3-3.2.10.tar.gz",
"sha256": "f87be7b4dec66db4098e9c167b2aa34e2ca10aeb5443bdde95ae03185ed513e0"
"url": "https://libsdl.org/release/SDL3-3.2.16.tar.gz",
"sha256": "6340e58879b2d15830c8460d2f589a385c444d1faa2a4828a9626c7322562be8"
}
],
"cleanup": [

View File

@@ -15,24 +15,24 @@
{
"type": "git",
"url": "https://github.com/google/shaderc.git",
"commit": "47a9387ef5b3600d30d84c71ec77a59dc7db46fa"
"commit": "8c2e602ce440b7739c95ff3d69cecb1adf6becda"
},
{
"type": "archive",
"url": "https://github.com/KhronosGroup/glslang/archive/142052fa30f9eca191aa9dcf65359fcaed09eeec.tar.gz",
"sha256": "aa27e4454ce631c5a17924ce0624eac736da19fc6f5a2ab15a6c58da7b36950f",
"url": "https://github.com/KhronosGroup/glslang/archive/efd24d75bcbc55620e759f6bf42c45a32abac5f8.tar.gz",
"sha256": "9427deccbdf4bde6a269938df38c6bd75247493786a310d8d733a2c82065ef47",
"dest": "third_party/glslang"
},
{
"type": "archive",
"url": "https://github.com/KhronosGroup/SPIRV-Headers/archive/5e3ad389ee56fca27c9705d093ae5387ce404df4.tar.gz",
"sha256": "5d866ce34a4b6908e262e5ebfffc0a5e11dd411640b5f24c85a80ad44c0d4697",
"url": "https://github.com/KhronosGroup/SPIRV-Headers/archive/2a611a970fdbc41ac2e3e328802aed9985352dca.tar.gz",
"sha256": "c2225a49c3d7efa5c4f4ce4a6b42081e6ea3daca376f3353d9d7c2722d77a28a",
"dest": "third_party/spirv-headers"
},
{
"type": "archive",
"url": "https://github.com/KhronosGroup/SPIRV-Tools/archive/dd4b663e13c07fea4fbb3f70c1c91c86731099f7.tar.gz",
"sha256": "03ee1a2c06f3b61008478f4abe9423454e53e580b9488b47c8071547c6a9db47",
"url": "https://github.com/KhronosGroup/SPIRV-Tools/archive/33e02568181e3312f49a3cf33df470bf96ef293a.tar.gz",
"sha256": "44d1005880c583fc00a0fb41c839214c68214b000ea8dcb54d352732fee600ff",
"dest": "third_party/spirv-tools"
},
{

View File

@@ -11,10 +11,10 @@
},
"sources": [
{
"type": "git",
"type": "git",
"url": "https://github.com/sammycage/plutovg.git",
"tag": "v0.0.13",
"sha256": "5e4712cf873b0c7829a4a6157763e2ad3ac49164"
"tag": "v1.1.0",
"commit": "1a8412d0574c4345dd7ef8a91ce7b58c7dcfe253"
}
],
"cleanup": [

View File

@@ -12,10 +12,10 @@
},
"sources": [
{
"type": "git",
"type": "git",
"url": "https://github.com/sammycage/plutosvg.git",
"tag": "v0.0.6",
"sha256": "c5388fa96feca1f1376a3d0485d5e35159452707"
"tag": "v0.0.7",
"commit": "31f7d2675416cd777c8e86220b035364873b2a8b"
}
],
"cleanup": [

View File

@@ -12,12 +12,24 @@ GIT_DATE=$(git log -1 --pretty=%cd --date=iso8601)
GIT_VERSION=$(git tag --points-at HEAD)
GIT_HASH=$(git rev-parse HEAD)
if [[ "${GIT_VERSION}" == "" ]]; then
# In the odd event that we run this script before the release gets tagged.
GIT_VERSION=$(git describe --tags)
if [[ "${GIT_VERSION}" == "" ]]; then
GIT_VERSION=$(git rev-parse HEAD)
fi
if [[ -z "${GIT_VERSION}" ]]; then
if git branch -r --contains HEAD | grep -q 'origin/master'; then
# Our master doesn't have a tagged commit
# This happens when the commit is "ci skip"
# abbrev so we have just the latest tag
# ie v2.3.420 (Yes, that's the current master at the time of writing)
GIT_VERSION=$(git describe --tags --abbrev=0)
else
# We are probably building a PR
# Keep the short SHA in the version
# ie v2.3.420-1-g10dc1a2da
GIT_VERSION=$(git describe --tags)
fi
if [[ -z "${GIT_VERSION}" ]]; then
# Fallback to raw commit hash
GIT_VERSION=$(git rev-parse HEAD)
fi
fi
echo "GIT_DATE: ${GIT_DATE}"

View File

@@ -11,7 +11,7 @@ merge_binaries() {
"
pushd "$X86DIR"
for X86BIN in $(find . -type f \( -name '*.dylib' -o -name '*.a' -o -perm +111 \)); do
if file "$X86DIR/$X86BIN" | grep "Mach-O " >/dev/null; then
if file "$X86DIR/$X86BIN" | grep "Mach-O.*x86_64" >/dev/null; then
ARMBIN="${ARMDIR}/${X86BIN}"
echo "Merge $ARMBIN to $X86BIN..."
lipo -create "$X86BIN" "$ARMBIN" -o "$X86BIN"
@@ -39,24 +39,24 @@ if [ "${INSTALLDIR:0:1}" != "/" ]; then
fi
FREETYPE=2.13.3
HARFBUZZ=10.0.1
SDL=SDL3-3.2.10
HARFBUZZ=11.2.0
SDL=SDL3-3.2.16
ZSTD=1.5.7
LZ4=b8fd2d15309dd4e605070bd4486e26b6ef814e29
LIBPNG=1.6.45
LZ4=1.10.0
LIBPNG=1.6.48
LIBJPEGTURBO=3.1.0
LIBWEBP=1.5.0
FFMPEG=6.0
MOLTENVK=1.2.9
QT=6.7.3
KDDOCKWIDGETS=2.2.3
PLUTOVG=0.0.13
PLUTOSVG=0.0.6
PLUTOVG=1.1.0
PLUTOSVG=0.0.7
SHADERC=2024.1
SHADERC_GLSLANG=142052fa30f9eca191aa9dcf65359fcaed09eeec
SHADERC_SPIRVHEADERS=5e3ad389ee56fca27c9705d093ae5387ce404df4
SHADERC_SPIRVTOOLS=dd4b663e13c07fea4fbb3f70c1c91c86731099f7
SHADERC=2025.3
SHADERC_GLSLANG=efd24d75bcbc55620e759f6bf42c45a32abac5f8
SHADERC_SPIRVHEADERS=2a611a970fdbc41ac2e3e328802aed9985352dca
SHADERC_SPIRVTOOLS=33e02568181e3312f49a3cf33df470bf96ef293a
mkdir -p deps-build
cd deps-build
@@ -78,11 +78,11 @@ CMAKE_ARCH_UNIVERSAL=-DCMAKE_OSX_ARCHITECTURES="x86_64;arm64"
cat > SHASUMS <<EOF
0550350666d427c74daeb85d5ac7bb353acba5f76956395995311a9c6f063289 freetype-$FREETYPE.tar.xz
e7358ea86fe10fb9261931af6f010d4358dac64f7074420ca9bc94aae2bdd542 harfbuzz-$HARFBUZZ.tar.gz
f87be7b4dec66db4098e9c167b2aa34e2ca10aeb5443bdde95ae03185ed513e0 $SDL.tar.gz
16c0204704f3ebeed057aba100fe7db18d71035505cb10e595ea33d346457fc8 harfbuzz-$HARFBUZZ.tar.gz
6340e58879b2d15830c8460d2f589a385c444d1faa2a4828a9626c7322562be8 $SDL.tar.gz
eb33e51f49a15e023950cd7825ca74a4a2b43db8354825ac24fc1b7ee09e6fa3 zstd-$ZSTD.tar.gz
0728800155f3ed0a0c87e03addbd30ecbe374f7b080678bbca1506051d50dec3 $LZ4.tar.gz
926485350139ffb51ef69760db35f78846c805fef3d59bfdcb2fba704663f370 libpng-$LIBPNG.tar.xz
537512904744b35e232912055ccf8ec66d768639ff3abe5788d90d792ec5f48b lz4-$LZ4.tar.gz
46fd06ff37db1db64c0dc288d78a3f5efd23ad9ac41561193f983e20937ece03 libpng-$LIBPNG.tar.xz
7d6fab70cf844bf6769077bd5d7a74893f8ffd4dfb42861745750c63c2a5c92c libwebp-$LIBWEBP.tar.gz
9564c72b1dfd1d6fe6274c5f95a8d989b59854575d4bbee44ade7bc17aa9bc93 libjpeg-turbo-$LIBJPEGTURBO.tar.gz
57be87c22d9b49c112b6d24bc67d42508660e6b718b3db89c44e47e289137082 ffmpeg-$FFMPEG.tar.xz
@@ -92,13 +92,13 @@ f415a09385030c6510a936155ce211f617c31506db5fbc563e804345f1ecf56e v$MOLTENVK.tar
40142cb71fb1e07ad612bc361b67f5d54cd9367f9979ae6b86124a064deda06b qtsvg-everywhere-src-$QT.tar.xz
f03bb7df619cd9ac9dba110e30b7bcab5dd88eb8bdc9cc752563b4367233203f qttools-everywhere-src-$QT.tar.xz
dcc762acac043b9bb5e4d369b6d6f53e0ecfcf76a408fe0db5f7ef071c9d6dc8 qttranslations-everywhere-src-$QT.tar.xz
eb3b5f0c16313d34f208d90c2fa1e588a23283eed63b101edd5422be6165d528 shaderc-$SHADERC.tar.gz
aa27e4454ce631c5a17924ce0624eac736da19fc6f5a2ab15a6c58da7b36950f shaderc-glslang-$SHADERC_GLSLANG.tar.gz
5d866ce34a4b6908e262e5ebfffc0a5e11dd411640b5f24c85a80ad44c0d4697 shaderc-spirv-headers-$SHADERC_SPIRVHEADERS.tar.gz
03ee1a2c06f3b61008478f4abe9423454e53e580b9488b47c8071547c6a9db47 shaderc-spirv-tools-$SHADERC_SPIRVTOOLS.tar.gz
a8e4a25e5c2686fd36981e527ed05e451fcfc226bddf350f4e76181371190937 shaderc-$SHADERC.tar.gz
9427deccbdf4bde6a269938df38c6bd75247493786a310d8d733a2c82065ef47 shaderc-glslang-$SHADERC_GLSLANG.tar.gz
c2225a49c3d7efa5c4f4ce4a6b42081e6ea3daca376f3353d9d7c2722d77a28a shaderc-spirv-headers-$SHADERC_SPIRVHEADERS.tar.gz
44d1005880c583fc00a0fb41c839214c68214b000ea8dcb54d352732fee600ff shaderc-spirv-tools-$SHADERC_SPIRVTOOLS.tar.gz
b8529755b2d54205341766ae168e83177c6120660539f9afba71af6bca4b81ec KDDockWidgets-$KDDOCKWIDGETS.tar.gz
f49d62709d6bf1808ddc9b8f71e22a755484f75c7bbb0fb368f7fb2ffc7cf645 plutovg-$PLUTOVG.tar.gz
01f8aee511bd587a602a166642a96522cc9522efd1e38c2d00e4fbc0aa22d7a0 plutosvg-$PLUTOSVG.tar.gz
8aa9860519c407890668c29998e8bb88896ef6a2e6d7ce5ac1e57f18d79e1525 plutovg-$PLUTOVG.tar.gz
78561b571ac224030cdc450ca2986b4de915c2ba7616004a6d71a379bffd15f3 plutosvg-$PLUTOSVG.tar.gz
EOF
curl -C - -L \
@@ -106,17 +106,17 @@ curl -C - -L \
-o "harfbuzz-$HARFBUZZ.tar.gz" "https://github.com/harfbuzz/harfbuzz/archive/refs/tags/$HARFBUZZ.tar.gz" \
-O "https://libsdl.org/release/$SDL.tar.gz" \
-O "https://github.com/facebook/zstd/releases/download/v$ZSTD/zstd-$ZSTD.tar.gz" \
-O "https://github.com/lz4/lz4/archive/$LZ4.tar.gz" \
-O "https://github.com/lz4/lz4/releases/download/v$LZ4/lz4-$LZ4.tar.gz" \
-O "https://downloads.sourceforge.net/project/libpng/libpng16/$LIBPNG/libpng-$LIBPNG.tar.xz" \
-O "https://github.com/libjpeg-turbo/libjpeg-turbo/releases/download/$LIBJPEGTURBO/libjpeg-turbo-$LIBJPEGTURBO.tar.gz" \
-O "https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-$LIBWEBP.tar.gz" \
-O "https://ffmpeg.org/releases/ffmpeg-$FFMPEG.tar.xz" \
-O "https://github.com/KhronosGroup/MoltenVK/archive/refs/tags/v$MOLTENVK.tar.gz" \
-O "https://download.qt.io/official_releases/qt/${QT%.*}/$QT/submodules/qtbase-everywhere-src-$QT.tar.xz" \
-O "https://download.qt.io/official_releases/qt/${QT%.*}/$QT/submodules/qtimageformats-everywhere-src-$QT.tar.xz" \
-O "https://download.qt.io/official_releases/qt/${QT%.*}/$QT/submodules/qtsvg-everywhere-src-$QT.tar.xz" \
-O "https://download.qt.io/official_releases/qt/${QT%.*}/$QT/submodules/qttools-everywhere-src-$QT.tar.xz" \
-O "https://download.qt.io/official_releases/qt/${QT%.*}/$QT/submodules/qttranslations-everywhere-src-$QT.tar.xz" \
-O "https://download.qt.io/archive/qt/${QT%.*}/$QT/submodules/qtbase-everywhere-src-$QT.tar.xz" \
-O "https://download.qt.io/archive/qt/${QT%.*}/$QT/submodules/qtimageformats-everywhere-src-$QT.tar.xz" \
-O "https://download.qt.io/archive/qt/${QT%.*}/$QT/submodules/qtsvg-everywhere-src-$QT.tar.xz" \
-O "https://download.qt.io/archive/qt/${QT%.*}/$QT/submodules/qttools-everywhere-src-$QT.tar.xz" \
-O "https://download.qt.io/archive/qt/${QT%.*}/$QT/submodules/qttranslations-everywhere-src-$QT.tar.xz" \
-o "shaderc-$SHADERC.tar.gz" "https://github.com/google/shaderc/archive/refs/tags/v$SHADERC.tar.gz" \
-o "shaderc-glslang-$SHADERC_GLSLANG.tar.gz" "https://github.com/KhronosGroup/glslang/archive/$SHADERC_GLSLANG.tar.gz" \
-o "shaderc-spirv-headers-$SHADERC_SPIRVHEADERS.tar.gz" "https://github.com/KhronosGroup/SPIRV-Headers/archive/$SHADERC_SPIRVHEADERS.tar.gz" \
@@ -187,7 +187,7 @@ cd ..
echo "Installing LZ4..."
rm -fr "lz4-$LZ4"
tar xf "$LZ4.tar.gz"
tar xf "lz4-$LZ4.tar.gz"
cd "lz4-$LZ4"
cmake "${CMAKE_COMMON[@]}" "$CMAKE_ARCH_X64" -DBUILD_SHARED_LIBS=ON -DLZ4_BUILD_CLI=OFF -DLZ4_BUILD_LEGACY_LZ4C=OFF -B build-dir build/cmake
make -C build-dir "-j$NPROCS"
@@ -213,8 +213,11 @@ echo "Installing libjpegturbo..."
rm -fr "libjpeg-turbo-$LIBJPEGTURBO"
tar xf "libjpeg-turbo-$LIBJPEGTURBO.tar.gz"
cd "libjpeg-turbo-$LIBJPEGTURBO"
cmake "${CMAKE_COMMON[@]}" "$CMAKE_ARCH_ARM64" -DENABLE_STATIC=OFF -DENABLE_SHARED=ON -B build-arm64
cmake "${CMAKE_COMMON[@]}" "$CMAKE_ARCH_X64" -DENABLE_STATIC=OFF -DENABLE_SHARED=ON -B build
make -C build "-j$NPROCS"
cmake "${CMAKE_COMMON[@]}" "$CMAKE_ARCH_ARM64" -DENABLE_STATIC=OFF -DENABLE_SHARED=ON -B build-arm64
make -C build-arm64 "-j$NPROCS"
merge_binaries $(realpath build) $(realpath build-arm64)
make -C build install
cd ..
@@ -380,7 +383,7 @@ echo "Building PlutoVG..."
rm -fr "plutovg-$PLUTOVG"
tar xf "plutovg-$PLUTOVG.tar.gz"
cd "plutovg-$PLUTOVG"
cmake "${CMAKE_COMMON[@]}" -DBUILD_SHARED_LIBS=ON -DPLUTOVG_BUILD_EXAMPLES=OFF -B build
cmake "${CMAKE_COMMON[@]}" "$CMAKE_ARCH_UNIVERSAL" -DBUILD_SHARED_LIBS=ON -DPLUTOVG_BUILD_EXAMPLES=OFF -B build
make -C build "-j$NPROCS"
make -C build install
cd ..
@@ -389,7 +392,7 @@ echo "Building PlutoSVG..."
rm -fr "plutosvg-$PLUTOSVG"
tar xf "plutosvg-$PLUTOSVG.tar.gz"
cd "plutosvg-$PLUTOSVG"
cmake "${CMAKE_COMMON[@]}" -DBUILD_SHARED_LIBS=ON -DPLUTOSVG_ENABLE_FREETYPE=ON -DPLUTOSVG_BUILD_EXAMPLES=OFF -B build
cmake "${CMAKE_COMMON[@]}" "$CMAKE_ARCH_UNIVERSAL" -DBUILD_SHARED_LIBS=ON -DPLUTOSVG_ENABLE_FREETYPE=ON -DPLUTOSVG_BUILD_EXAMPLES=OFF -B build
make -C build "-j$NPROCS"
make -C build install
cd ..

View File

@@ -21,24 +21,24 @@ if [ "${INSTALLDIR:0:1}" != "/" ]; then
fi
FREETYPE=2.13.3
HARFBUZZ=10.0.1
SDL=SDL3-3.2.10
HARFBUZZ=11.2.0
SDL=SDL3-3.2.16
ZSTD=1.5.7
LZ4=b8fd2d15309dd4e605070bd4486e26b6ef814e29
LIBPNG=1.6.45
LZ4=1.10.0
LIBPNG=1.6.48
LIBJPEGTURBO=3.1.0
LIBWEBP=1.5.0
FFMPEG=6.0
MOLTENVK=1.2.9
QT=6.7.3
KDDOCKWIDGETS=2.2.3
PLUTOVG=0.0.13
PLUTOSVG=0.0.6
PLUTOVG=1.1.0
PLUTOSVG=0.0.7
SHADERC=2024.1
SHADERC_GLSLANG=142052fa30f9eca191aa9dcf65359fcaed09eeec
SHADERC_SPIRVHEADERS=5e3ad389ee56fca27c9705d093ae5387ce404df4
SHADERC_SPIRVTOOLS=dd4b663e13c07fea4fbb3f70c1c91c86731099f7
SHADERC=2025.3
SHADERC_GLSLANG=efd24d75bcbc55620e759f6bf42c45a32abac5f8
SHADERC_SPIRVHEADERS=2a611a970fdbc41ac2e3e328802aed9985352dca
SHADERC_SPIRVTOOLS=33e02568181e3312f49a3cf33df470bf96ef293a
mkdir -p deps-build
cd deps-build
@@ -58,11 +58,11 @@ CMAKE_COMMON=(
cat > SHASUMS <<EOF
0550350666d427c74daeb85d5ac7bb353acba5f76956395995311a9c6f063289 freetype-$FREETYPE.tar.xz
e7358ea86fe10fb9261931af6f010d4358dac64f7074420ca9bc94aae2bdd542 harfbuzz-$HARFBUZZ.tar.gz
f87be7b4dec66db4098e9c167b2aa34e2ca10aeb5443bdde95ae03185ed513e0 $SDL.tar.gz
16c0204704f3ebeed057aba100fe7db18d71035505cb10e595ea33d346457fc8 harfbuzz-$HARFBUZZ.tar.gz
6340e58879b2d15830c8460d2f589a385c444d1faa2a4828a9626c7322562be8 $SDL.tar.gz
eb33e51f49a15e023950cd7825ca74a4a2b43db8354825ac24fc1b7ee09e6fa3 zstd-$ZSTD.tar.gz
0728800155f3ed0a0c87e03addbd30ecbe374f7b080678bbca1506051d50dec3 $LZ4.tar.gz
926485350139ffb51ef69760db35f78846c805fef3d59bfdcb2fba704663f370 libpng-$LIBPNG.tar.xz
537512904744b35e232912055ccf8ec66d768639ff3abe5788d90d792ec5f48b lz4-$LZ4.tar.gz
46fd06ff37db1db64c0dc288d78a3f5efd23ad9ac41561193f983e20937ece03 libpng-$LIBPNG.tar.xz
7d6fab70cf844bf6769077bd5d7a74893f8ffd4dfb42861745750c63c2a5c92c libwebp-$LIBWEBP.tar.gz
9564c72b1dfd1d6fe6274c5f95a8d989b59854575d4bbee44ade7bc17aa9bc93 libjpeg-turbo-$LIBJPEGTURBO.tar.gz
57be87c22d9b49c112b6d24bc67d42508660e6b718b3db89c44e47e289137082 ffmpeg-$FFMPEG.tar.xz
@@ -72,13 +72,13 @@ f415a09385030c6510a936155ce211f617c31506db5fbc563e804345f1ecf56e v$MOLTENVK.tar
40142cb71fb1e07ad612bc361b67f5d54cd9367f9979ae6b86124a064deda06b qtsvg-everywhere-src-$QT.tar.xz
f03bb7df619cd9ac9dba110e30b7bcab5dd88eb8bdc9cc752563b4367233203f qttools-everywhere-src-$QT.tar.xz
dcc762acac043b9bb5e4d369b6d6f53e0ecfcf76a408fe0db5f7ef071c9d6dc8 qttranslations-everywhere-src-$QT.tar.xz
eb3b5f0c16313d34f208d90c2fa1e588a23283eed63b101edd5422be6165d528 shaderc-$SHADERC.tar.gz
aa27e4454ce631c5a17924ce0624eac736da19fc6f5a2ab15a6c58da7b36950f shaderc-glslang-$SHADERC_GLSLANG.tar.gz
5d866ce34a4b6908e262e5ebfffc0a5e11dd411640b5f24c85a80ad44c0d4697 shaderc-spirv-headers-$SHADERC_SPIRVHEADERS.tar.gz
03ee1a2c06f3b61008478f4abe9423454e53e580b9488b47c8071547c6a9db47 shaderc-spirv-tools-$SHADERC_SPIRVTOOLS.tar.gz
a8e4a25e5c2686fd36981e527ed05e451fcfc226bddf350f4e76181371190937 shaderc-$SHADERC.tar.gz
9427deccbdf4bde6a269938df38c6bd75247493786a310d8d733a2c82065ef47 shaderc-glslang-$SHADERC_GLSLANG.tar.gz
c2225a49c3d7efa5c4f4ce4a6b42081e6ea3daca376f3353d9d7c2722d77a28a shaderc-spirv-headers-$SHADERC_SPIRVHEADERS.tar.gz
44d1005880c583fc00a0fb41c839214c68214b000ea8dcb54d352732fee600ff shaderc-spirv-tools-$SHADERC_SPIRVTOOLS.tar.gz
b8529755b2d54205341766ae168e83177c6120660539f9afba71af6bca4b81ec KDDockWidgets-$KDDOCKWIDGETS.tar.gz
f49d62709d6bf1808ddc9b8f71e22a755484f75c7bbb0fb368f7fb2ffc7cf645 plutovg-$PLUTOVG.tar.gz
01f8aee511bd587a602a166642a96522cc9522efd1e38c2d00e4fbc0aa22d7a0 plutosvg-$PLUTOSVG.tar.gz
8aa9860519c407890668c29998e8bb88896ef6a2e6d7ce5ac1e57f18d79e1525 plutovg-$PLUTOVG.tar.gz
78561b571ac224030cdc450ca2986b4de915c2ba7616004a6d71a379bffd15f3 plutosvg-$PLUTOSVG.tar.gz
EOF
curl -L \
@@ -86,7 +86,7 @@ curl -L \
-o "harfbuzz-$HARFBUZZ.tar.gz" "https://github.com/harfbuzz/harfbuzz/archive/refs/tags/$HARFBUZZ.tar.gz" \
-O "https://libsdl.org/release/$SDL.tar.gz" \
-O "https://github.com/facebook/zstd/releases/download/v$ZSTD/zstd-$ZSTD.tar.gz" \
-O "https://github.com/lz4/lz4/archive/$LZ4.tar.gz" \
-O "https://github.com/lz4/lz4/releases/download/v$LZ4/lz4-$LZ4.tar.gz" \
-O "https://downloads.sourceforge.net/project/libpng/libpng16/$LIBPNG/libpng-$LIBPNG.tar.xz" \
-O "https://github.com/libjpeg-turbo/libjpeg-turbo/releases/download/$LIBJPEGTURBO/libjpeg-turbo-$LIBJPEGTURBO.tar.gz" \
-O "https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-$LIBWEBP.tar.gz" \
@@ -146,7 +146,7 @@ cd ..
echo "Installing LZ4..."
rm -fr "lz4-$LZ4"
tar xf "$LZ4.tar.gz"
tar xf "lz4-$LZ4.tar.gz"
cd "lz4-$LZ4"
cmake "${CMAKE_COMMON[@]}" -DBUILD_SHARED_LIBS=ON -DLZ4_BUILD_CLI=OFF -DLZ4_BUILD_LEGACY_LZ4C=OFF -B build-dir build/cmake
make -C build-dir "-j$NPROCS"

View File

@@ -66,7 +66,7 @@ const embed = new MessageEmbed()
.setDescription("To download the latest or previous builds, [visit the official downloads page](https://pcsx2.net/downloads/).")
.addFields(
{ name: 'Version', value: releaseInfo.tag_name, inline: true },
{ name: 'Installation Steps', value: '[See Here](https://github.com/PCSX2/pcsx2/wiki/Nightly-Build-Usage-Guide)', inline: true },
{ name: 'Installation Steps', value: '[See Here](https://pcsx2.net/docs/category/setup)', inline: true },
{ name: 'Included Changes', value: releaseInfo.body, inline: false }
);
console.log(embed);

View File

@@ -43,48 +43,48 @@ echo INSTALLDIR=%INSTALLDIR%
cd "%BUILDDIR%"
set FREETYPE=2.13.3
set HARFBUZZ=10.0.1
set HARFBUZZ=11.2.0
set LIBJPEGTURBO=3.1.0
set LIBPNG=1645
set LZ4=b8fd2d15309dd4e605070bd4486e26b6ef814e29
set SDL=SDL3-3.2.10
set QT=6.9.0
set LIBPNG=1648
set SDL=SDL3-3.2.16
set QT=6.9.1
set QTMINOR=6.9
set LZ4=1.10.0
set WEBP=1.5.0
set ZLIB=1.3.1
set ZLIBSHORT=131
set ZSTD=1.5.7
set KDDOCKWIDGETS=2.2.3
set PLUTOVG=0.0.13
set PLUTOSVG=0.0.6
set PLUTOVG=1.1.0
set PLUTOSVG=0.0.7
set SHADERC=2024.1
set SHADERC_GLSLANG=142052fa30f9eca191aa9dcf65359fcaed09eeec
set SHADERC_SPIRVHEADERS=5e3ad389ee56fca27c9705d093ae5387ce404df4
set SHADERC_SPIRVTOOLS=dd4b663e13c07fea4fbb3f70c1c91c86731099f7
set SHADERC=2025.3
set SHADERC_GLSLANG=efd24d75bcbc55620e759f6bf42c45a32abac5f8
set SHADERC_SPIRVHEADERS=2a611a970fdbc41ac2e3e328802aed9985352dca
set SHADERC_SPIRVTOOLS=33e02568181e3312f49a3cf33df470bf96ef293a
call :downloadfile "freetype-%FREETYPE%.tar.gz" https://sourceforge.net/projects/freetype/files/freetype2/%FREETYPE%/freetype-%FREETYPE%.tar.gz/download 5c3a8e78f7b24c20b25b54ee575d6daa40007a5f4eea2845861c3409b3021747 || goto error
call :downloadfile "harfbuzz-%HARFBUZZ%.zip" https://github.com/harfbuzz/harfbuzz/archive/refs/tags/%HARFBUZZ%.zip 8adf9f5a4b6022aa2744f45c89ce347df46fea8403e99f01d650b11c417d0aa8 || goto error
call :downloadfile "lpng%LIBPNG%.zip" https://download.sourceforge.net/libpng/lpng1645.zip a66c4b1350b67776e90263e2550933067cd9ccbd318db489f84dcc0d2b033249 || goto error
call :downloadfile "harfbuzz-%HARFBUZZ%.zip" https://github.com/harfbuzz/harfbuzz/archive/refs/tags/%HARFBUZZ%.zip 850cb5e38e21106c0abba86c5b73f8f74b9a32d7725505901d081080b0d3f0b3 || goto error
call :downloadfile "lpng%LIBPNG%.zip" https://download.sourceforge.net/libpng/lpng1648.zip 2e5f080360f77376eb2bfa9e2ed773b9c7728159aba47b638ad53ca839379040 || goto error
call :downloadfile "libjpeg-turbo-%LIBJPEGTURBO%.tar.gz" "https://github.com/libjpeg-turbo/libjpeg-turbo/releases/download/%LIBJPEGTURBO%/libjpeg-turbo-%LIBJPEGTURBO%.tar.gz" 9564c72b1dfd1d6fe6274c5f95a8d989b59854575d4bbee44ade7bc17aa9bc93 || goto error
call :downloadfile "libwebp-%WEBP%.tar.gz" "https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-%WEBP%.tar.gz" 7d6fab70cf844bf6769077bd5d7a74893f8ffd4dfb42861745750c63c2a5c92c || goto error
call :downloadfile "lz4-%LZ4%.zip" "https://github.com/lz4/lz4/archive/%LZ4%.zip" 0c33119688d6b180c7e760b0acd70059222389cfd581632623784bee27e51a31 || goto error
call :downloadfile "%SDL%.zip" "https://libsdl.org/release/%SDL%.zip" 01d9ab20fc071b076be91df5396b464b4ef159e93b2b2addda1cc36750fc1f29 || goto error
call :downloadfile "qtbase-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qtbase-everywhere-src-%QT%.zip" 513df15a6365a40f6230ec9463ad8c71b824e181d4b661dac9707e103b24ae0c || goto error
call :downloadfile "qtimageformats-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qtimageformats-everywhere-src-%QT%.zip" d428fd17a0d3f92c48a30f1d23806bf20352fbce2e80e5bbee27fa80576480ee || goto error
call :downloadfile "qtsvg-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qtsvg-everywhere-src-%QT%.zip" 54bf06afeb67035f1c6afcd00beec755c0d776626b4cce9ab56992a55215ba69 || goto error
call :downloadfile "qttools-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qttools-everywhere-src-%QT%.zip" 5f8a94a161bd2e71a82f478dc19f4ec77ac95a50709f5a68d5951001ed6bb856 || goto error
call :downloadfile "qttranslations-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qttranslations-everywhere-src-%QT%.zip" 5885ce1a114615cc5fa69e459f069d3fe2bcb1320fd9cc162821f3920ef44735 || goto error
call :downloadfile "%SDL%.zip" "https://libsdl.org/release/%SDL%.zip" 0cc7430fb827c1f843e31b8b26ba7f083b1eeb8f6315a65d3744fd4d25b6c373 || goto error
call :downloadfile "qtbase-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qtbase-everywhere-src-%QT%.zip" efa6d8ef9f7ae0fd9f7d280fbff574d71882b60a357ae639e516dc173cf26986 || goto error
call :downloadfile "qtimageformats-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qtimageformats-everywhere-src-%QT%.zip" 8439d3394bc380fd17a920ee96df1d2272bf8d3490871d948ef750f95e0ded06 || goto error
call :downloadfile "qtsvg-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qtsvg-everywhere-src-%QT%.zip" a8f90c768b54e28d61e02c1229b74a2b834e9852af523e5c70bcd2ae4c34a772 || goto error
call :downloadfile "qttools-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qttools-everywhere-src-%QT%.zip" 38db91c4a8044c395eac89e325ecc25edbda12606fc28812491ef5e5b6b53dd6 || goto error
call :downloadfile "qttranslations-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qttranslations-everywhere-src-%QT%.zip" fd2e776164751fb486495efeee336d26d85fe1ca1f6a7b9eb6aafca2e3d333aa || goto error
call :downloadfile "lz4-%LZ4%.zip" "https://github.com/lz4/lz4/archive/refs/tags/v%LZ4%.zip" 3224b4c80f351f194984526ef396f6079bd6332dd9825c72ac0d7a37b3cdc565 || goto error
call :downloadfile "zlib%ZLIBSHORT%.zip" "https://zlib.net/zlib%ZLIBSHORT%.zip" 72af66d44fcc14c22013b46b814d5d2514673dda3d115e64b690c1ad636e7b17 || goto error
call :downloadfile "zstd-%ZSTD%.zip" "https://github.com/facebook/zstd/archive/refs/tags/v%ZSTD%.zip" 7897bc5d620580d9b7cd3539c44b59d78f3657d33663fe97a145e07b4ebd69a4 || goto error
call :downloadfile "KDDockWidgets-%KDDOCKWIDGETS%.zip" "https://github.com/KDAB/KDDockWidgets/archive/v%KDDOCKWIDGETS%.zip" 1ba8e5b48f3b4d47d2de7121529d448532200fa36d9ed21f93909f6eb03f61cb || goto error
call :downloadfile "plutovg-%PLUTOVG%.zip" "https://github.com/sammycage/plutovg/archive/v%PLUTOVG%.zip" e313baaa7c934503ef601c909661a84e5b795dfa12f0354721cac7a9c27be47e || goto error
call :downloadfile "plutosvg-%PLUTOSVG%.zip" "https://github.com/sammycage/plutosvg/archive/v%PLUTOSVG%.zip" 24826a70d0b168a66eb16ec9d7eeeba0d4ca9d4babc1199889d374918008426e || goto error
call :downloadfile "plutovg-%PLUTOVG%.zip" "https://github.com/sammycage/plutovg/archive/v%PLUTOVG%.zip" 83b2cd6230909a8d586518f49e79e4a1b1e9fab3847db6a678ec9d2dacab052a || goto error
call :downloadfile "plutosvg-%PLUTOSVG%.zip" "https://github.com/sammycage/plutosvg/archive/v%PLUTOSVG%.zip" 82dee2c57ad712bdd6d6d81d3e76249d89caa4b5a4214353660fd5adff12201a || goto error
call :downloadfile "shaderc-%SHADERC%.zip" "https://github.com/google/shaderc/archive/refs/tags/v%SHADERC%.zip" 6c9f42ed6bf42750f5369b089909abfdcf0101488b4a1f41116d5159d00af8e7 || goto error
call :downloadfile "shaderc-glslang-%SHADERC_GLSLANG%.zip" "https://github.com/KhronosGroup/glslang/archive/%SHADERC_GLSLANG%.zip" 03ad8a6fa987af4653d0cfe6bdaed41bcf617f1366a151fb1574da75950cd3e8 || goto error
call :downloadfile "shaderc-spirv-headers-%SHADERC_SPIRVHEADERS%.zip" "https://github.com/KhronosGroup/SPIRV-Headers/archive/%SHADERC_SPIRVHEADERS%.zip" fa59a54334feaba5702b9c25724c3f4746123865769b36dd5a28d9ef5e9d39ab || goto error
call :downloadfile "shaderc-spirv-tools-%SHADERC_SPIRVTOOLS%.zip" "https://github.com/KhronosGroup/SPIRV-Tools/archive/%SHADERC_SPIRVTOOLS%.zip" bf385994c20293485b378c27dfdbd77a31b949deabccd9218a977f173eda9f6f || goto error
call :downloadfile "shaderc-%SHADERC%.zip" "https://github.com/google/shaderc/archive/refs/tags/v%SHADERC%.zip" 77d2425458bca62c16b1ed49ed02de4c4114a113781bd94c1961b273bdca00fb || goto error
call :downloadfile "shaderc-glslang-%SHADERC_GLSLANG%.zip" "https://github.com/KhronosGroup/glslang/archive/%SHADERC_GLSLANG%.zip" ebd389bf79c17d79d999b3e9756359945020bbef799537aa96d8900464c373c5 || goto error
call :downloadfile "shaderc-spirv-headers-%SHADERC_SPIRVHEADERS%.zip" "https://github.com/KhronosGroup/SPIRV-Headers/archive/%SHADERC_SPIRVHEADERS%.zip" 6b954cb358a43915a54b6ca7a27db11b15c4f6e9ec547ab4cad71857354692bc || goto error
call :downloadfile "shaderc-spirv-tools-%SHADERC_SPIRVTOOLS%.zip" "https://github.com/KhronosGroup/SPIRV-Tools/archive/%SHADERC_SPIRVTOOLS%.zip" 00c4fa1a26de21c7c8db6947e06094a338e7d4edf972bc70d30afea9315373f2 || goto error
if %DEBUG%==1 (
echo Building debug and release libraries...
@@ -246,12 +246,21 @@ cmake --build . --parallel || goto error
ninja install || goto error
cd ..\.. || goto error
if %DEBUG%==1 (
set KDDOCKWIDGETSBUILDSPEC=-DCMAKE_CONFIGURATION_TYPES="Release;Debug" -DCMAKE_CROSS_CONFIGS=all -DCMAKE_DEFAULT_BUILD_TYPE=Release -DCMAKE_DEFAULT_CONFIGS=all -G "Ninja Multi-Config"
) else (
rem kddockwidgets slightly changes the name of the dll depending on if CMAKE_BUILD_TYPE or CMAKE_CONFIGURATION_TYPES is used
rem The dll name being kddockwidgets-qt62.dll or kddockwidgets-qt62.dll respectively
rem Always use CMAKE_CONFIGURATION_TYPES to give consistant naming
set KDDOCKWIDGETSBUILDSPEC=-DCMAKE_CONFIGURATION_TYPES=Release -DCMAKE_CROSS_CONFIGS=all -DCMAKE_DEFAULT_BUILD_TYPE=Release -DCMAKE_DEFAULT_CONFIGS=Release -G "Ninja Multi-Config"
)
echo "Building KDDockWidgets..."
rmdir /S /Q "KDDockWidgets-%KDDOCKWIDGETS%"
%SEVENZIP% x "KDDockWidgets-%KDDOCKWIDGETS%.zip" || goto error
cd "KDDockWidgets-%KDDOCKWIDGETS%" || goto error
%PATCH% -p1 < "%SCRIPTDIR%\..\common\kddockwidgets-dodgy-include.patch" || goto error
cmake %ARM64TOOLCHAIN% -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH="%INSTALLDIR%" -DCMAKE_INSTALL_PREFIX="%INSTALLDIR%" -DKDDockWidgets_QT6=true -DKDDockWidgets_EXAMPLES=false -DKDDockWidgets_FRONTENDS=qtwidgets -B build -G Ninja || goto error
cmake -B build %ARM64TOOLCHAIN% -DCMAKE_PREFIX_PATH="%INSTALLDIR%" -DCMAKE_INSTALL_PREFIX="%INSTALLDIR%" -DKDDockWidgets_QT6=true -DKDDockWidgets_EXAMPLES=false -DKDDockWidgets_FRONTENDS=qtwidgets %KDDOCKWIDGETSBUILDSPEC% || goto error
cmake --build build --parallel || goto error
ninja -C build install || goto error
cd .. || goto error

View File

@@ -41,48 +41,48 @@ set "PATH=%PATH%;%INSTALLDIR%\bin"
cd "%BUILDDIR%"
set FREETYPE=2.13.3
set HARFBUZZ=10.0.1
set HARFBUZZ=11.2.0
set LIBJPEGTURBO=3.1.0
set LIBPNG=1645
set LZ4=b8fd2d15309dd4e605070bd4486e26b6ef814e29
set SDL=SDL3-3.2.10
set QT=6.9.0
set LIBPNG=1648
set SDL=SDL3-3.2.16
set QT=6.9.1
set QTMINOR=6.9
set LZ4=1.10.0
set WEBP=1.5.0
set ZLIB=1.3.1
set ZLIBSHORT=131
set ZSTD=1.5.7
set KDDOCKWIDGETS=2.2.3
set PLUTOVG=0.0.13
set PLUTOSVG=0.0.6
set PLUTOVG=1.1.0
set PLUTOSVG=0.0.7
set SHADERC=2024.1
set SHADERC_GLSLANG=142052fa30f9eca191aa9dcf65359fcaed09eeec
set SHADERC_SPIRVHEADERS=5e3ad389ee56fca27c9705d093ae5387ce404df4
set SHADERC_SPIRVTOOLS=dd4b663e13c07fea4fbb3f70c1c91c86731099f7
set SHADERC=2025.3
set SHADERC_GLSLANG=efd24d75bcbc55620e759f6bf42c45a32abac5f8
set SHADERC_SPIRVHEADERS=2a611a970fdbc41ac2e3e328802aed9985352dca
set SHADERC_SPIRVTOOLS=33e02568181e3312f49a3cf33df470bf96ef293a
call :downloadfile "freetype-%FREETYPE%.tar.gz" https://sourceforge.net/projects/freetype/files/freetype2/%FREETYPE%/freetype-%FREETYPE%.tar.gz/download 5c3a8e78f7b24c20b25b54ee575d6daa40007a5f4eea2845861c3409b3021747 || goto error
call :downloadfile "harfbuzz-%HARFBUZZ%.zip" https://github.com/harfbuzz/harfbuzz/archive/refs/tags/%HARFBUZZ%.zip 8adf9f5a4b6022aa2744f45c89ce347df46fea8403e99f01d650b11c417d0aa8 || goto error
call :downloadfile "lpng%LIBPNG%.zip" https://download.sourceforge.net/libpng/lpng1645.zip a66c4b1350b67776e90263e2550933067cd9ccbd318db489f84dcc0d2b033249 || goto error
call :downloadfile "harfbuzz-%HARFBUZZ%.zip" https://github.com/harfbuzz/harfbuzz/archive/refs/tags/%HARFBUZZ%.zip 850cb5e38e21106c0abba86c5b73f8f74b9a32d7725505901d081080b0d3f0b3 || goto error
call :downloadfile "lpng%LIBPNG%.zip" https://download.sourceforge.net/libpng/lpng1648.zip 2e5f080360f77376eb2bfa9e2ed773b9c7728159aba47b638ad53ca839379040 || goto error
call :downloadfile "libjpeg-turbo-%LIBJPEGTURBO%.tar.gz" "https://github.com/libjpeg-turbo/libjpeg-turbo/releases/download/%LIBJPEGTURBO%/libjpeg-turbo-%LIBJPEGTURBO%.tar.gz" 9564c72b1dfd1d6fe6274c5f95a8d989b59854575d4bbee44ade7bc17aa9bc93 || goto error
call :downloadfile "libwebp-%WEBP%.tar.gz" "https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-%WEBP%.tar.gz" 7d6fab70cf844bf6769077bd5d7a74893f8ffd4dfb42861745750c63c2a5c92c || goto error
call :downloadfile "lz4-%LZ4%.zip" "https://github.com/lz4/lz4/archive/%LZ4%.zip" 0c33119688d6b180c7e760b0acd70059222389cfd581632623784bee27e51a31 || goto error
call :downloadfile "%SDL%.zip" "https://libsdl.org/release/%SDL%.zip" 01d9ab20fc071b076be91df5396b464b4ef159e93b2b2addda1cc36750fc1f29 || goto error
call :downloadfile "qtbase-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qtbase-everywhere-src-%QT%.zip" 513df15a6365a40f6230ec9463ad8c71b824e181d4b661dac9707e103b24ae0c || goto error
call :downloadfile "qtimageformats-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qtimageformats-everywhere-src-%QT%.zip" d428fd17a0d3f92c48a30f1d23806bf20352fbce2e80e5bbee27fa80576480ee || goto error
call :downloadfile "qtsvg-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qtsvg-everywhere-src-%QT%.zip" 54bf06afeb67035f1c6afcd00beec755c0d776626b4cce9ab56992a55215ba69 || goto error
call :downloadfile "qttools-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qttools-everywhere-src-%QT%.zip" 5f8a94a161bd2e71a82f478dc19f4ec77ac95a50709f5a68d5951001ed6bb856 || goto error
call :downloadfile "qttranslations-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qttranslations-everywhere-src-%QT%.zip" 5885ce1a114615cc5fa69e459f069d3fe2bcb1320fd9cc162821f3920ef44735 || goto error
call :downloadfile "%SDL%.zip" "https://libsdl.org/release/%SDL%.zip" 0cc7430fb827c1f843e31b8b26ba7f083b1eeb8f6315a65d3744fd4d25b6c373 || goto error
call :downloadfile "qtbase-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qtbase-everywhere-src-%QT%.zip" efa6d8ef9f7ae0fd9f7d280fbff574d71882b60a357ae639e516dc173cf26986 || goto error
call :downloadfile "qtimageformats-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qtimageformats-everywhere-src-%QT%.zip" 8439d3394bc380fd17a920ee96df1d2272bf8d3490871d948ef750f95e0ded06 || goto error
call :downloadfile "qtsvg-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qtsvg-everywhere-src-%QT%.zip" a8f90c768b54e28d61e02c1229b74a2b834e9852af523e5c70bcd2ae4c34a772 || goto error
call :downloadfile "qttools-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qttools-everywhere-src-%QT%.zip" 38db91c4a8044c395eac89e325ecc25edbda12606fc28812491ef5e5b6b53dd6 || goto error
call :downloadfile "qttranslations-everywhere-src-%QT%.zip" "https://download.qt.io/official_releases/qt/%QTMINOR%/%QT%/submodules/qttranslations-everywhere-src-%QT%.zip" fd2e776164751fb486495efeee336d26d85fe1ca1f6a7b9eb6aafca2e3d333aa || goto error
call :downloadfile "lz4-%LZ4%.zip" "https://github.com/lz4/lz4/archive/refs/tags/v%LZ4%.zip" 3224b4c80f351f194984526ef396f6079bd6332dd9825c72ac0d7a37b3cdc565 || goto error
call :downloadfile "zlib%ZLIBSHORT%.zip" "https://zlib.net/zlib%ZLIBSHORT%.zip" 72af66d44fcc14c22013b46b814d5d2514673dda3d115e64b690c1ad636e7b17 || goto error
call :downloadfile "zstd-%ZSTD%.zip" "https://github.com/facebook/zstd/archive/refs/tags/v%ZSTD%.zip" 7897bc5d620580d9b7cd3539c44b59d78f3657d33663fe97a145e07b4ebd69a4 || goto error
call :downloadfile "KDDockWidgets-%KDDOCKWIDGETS%.zip" "https://github.com/KDAB/KDDockWidgets/archive/v%KDDOCKWIDGETS%.zip" 1ba8e5b48f3b4d47d2de7121529d448532200fa36d9ed21f93909f6eb03f61cb || goto error
call :downloadfile "plutovg-%PLUTOVG%.zip" "https://github.com/sammycage/plutovg/archive/v%PLUTOVG%.zip" e313baaa7c934503ef601c909661a84e5b795dfa12f0354721cac7a9c27be47e || goto error
call :downloadfile "plutosvg-%PLUTOSVG%.zip" "https://github.com/sammycage/plutosvg/archive/v%PLUTOSVG%.zip" 24826a70d0b168a66eb16ec9d7eeeba0d4ca9d4babc1199889d374918008426e || goto error
call :downloadfile "plutovg-%PLUTOVG%.zip" "https://github.com/sammycage/plutovg/archive/v%PLUTOVG%.zip" 83b2cd6230909a8d586518f49e79e4a1b1e9fab3847db6a678ec9d2dacab052a || goto error
call :downloadfile "plutosvg-%PLUTOSVG%.zip" "https://github.com/sammycage/plutosvg/archive/v%PLUTOSVG%.zip" 82dee2c57ad712bdd6d6d81d3e76249d89caa4b5a4214353660fd5adff12201a || goto error
call :downloadfile "shaderc-%SHADERC%.zip" "https://github.com/google/shaderc/archive/refs/tags/v%SHADERC%.zip" 6c9f42ed6bf42750f5369b089909abfdcf0101488b4a1f41116d5159d00af8e7 || goto error
call :downloadfile "shaderc-glslang-%SHADERC_GLSLANG%.zip" "https://github.com/KhronosGroup/glslang/archive/%SHADERC_GLSLANG%.zip" 03ad8a6fa987af4653d0cfe6bdaed41bcf617f1366a151fb1574da75950cd3e8 || goto error
call :downloadfile "shaderc-spirv-headers-%SHADERC_SPIRVHEADERS%.zip" "https://github.com/KhronosGroup/SPIRV-Headers/archive/%SHADERC_SPIRVHEADERS%.zip" fa59a54334feaba5702b9c25724c3f4746123865769b36dd5a28d9ef5e9d39ab || goto error
call :downloadfile "shaderc-spirv-tools-%SHADERC_SPIRVTOOLS%.zip" "https://github.com/KhronosGroup/SPIRV-Tools/archive/%SHADERC_SPIRVTOOLS%.zip" bf385994c20293485b378c27dfdbd77a31b949deabccd9218a977f173eda9f6f || goto error
call :downloadfile "shaderc-%SHADERC%.zip" "https://github.com/google/shaderc/archive/refs/tags/v%SHADERC%.zip" 77d2425458bca62c16b1ed49ed02de4c4114a113781bd94c1961b273bdca00fb || goto error
call :downloadfile "shaderc-glslang-%SHADERC_GLSLANG%.zip" "https://github.com/KhronosGroup/glslang/archive/%SHADERC_GLSLANG%.zip" ebd389bf79c17d79d999b3e9756359945020bbef799537aa96d8900464c373c5 || goto error
call :downloadfile "shaderc-spirv-headers-%SHADERC_SPIRVHEADERS%.zip" "https://github.com/KhronosGroup/SPIRV-Headers/archive/%SHADERC_SPIRVHEADERS%.zip" 6b954cb358a43915a54b6ca7a27db11b15c4f6e9ec547ab4cad71857354692bc || goto error
call :downloadfile "shaderc-spirv-tools-%SHADERC_SPIRVTOOLS%.zip" "https://github.com/KhronosGroup/SPIRV-Tools/archive/%SHADERC_SPIRVTOOLS%.zip" 00c4fa1a26de21c7c8db6947e06094a338e7d4edf972bc70d30afea9315373f2 || goto error
if %DEBUG%==1 (
echo Building debug and release libraries...
@@ -90,7 +90,7 @@ if %DEBUG%==1 (
echo Building release libraries...
)
set FORCEPDB=-DCMAKE_SHARED_LINKER_FLAGS_RELEASE="/DEBUG"
set FORCEPDB=-DCMAKE_SHARED_LINKER_FLAGS_RELEASE="/DEBUG" -DCMAKE_MODULE_LINKER_FLAGS_RELEASE="/DEBUG"
echo Building Zlib...
rmdir /S /Q "zlib-%ZLIB%"
@@ -198,9 +198,6 @@ cd "qtbase-everywhere-src-%QT%" || goto error
rem Disable the PCRE2 JIT, it doesn't properly verify AVX2 support.
%PATCH% -p1 < "%SCRIPTDIR%\qtbase-disable-pcre2-jit.patch" || goto error
rem Hackfix settings icon stretching
%PATCH% -p1 < "%SCRIPTDIR%\qtbase-fix-icon-stretch.patch" || goto error
cmake -B build -DFEATURE_sql=OFF -DCMAKE_INSTALL_PREFIX="%INSTALLDIR%" %FORCEPDB% -DINPUT_gui=yes -DINPUT_widgets=yes -DINPUT_ssl=yes -DINPUT_openssl=no -DINPUT_schannel=yes -DFEATURE_system_png=ON -DFEATURE_system_jpeg=ON -DFEATURE_system_zlib=ON -DFEATURE_system_freetype=ON -DFEATURE_system_harfbuzz=ON %QTBUILDSPEC% || goto error
cmake --build build --parallel || goto error
ninja -C build install || goto error
@@ -250,12 +247,21 @@ cmake --build . --parallel || goto error
ninja install || goto error
cd ..\.. || goto error
if %DEBUG%==1 (
set KDDOCKWIDGETSBUILDSPEC=-DCMAKE_CONFIGURATION_TYPES="Release;Debug" -DCMAKE_CROSS_CONFIGS=all -DCMAKE_DEFAULT_BUILD_TYPE=Release -DCMAKE_DEFAULT_CONFIGS=all -G "Ninja Multi-Config"
) else (
rem kddockwidgets slightly changes the name of the dll depending on if CMAKE_BUILD_TYPE or CMAKE_CONFIGURATION_TYPES is used
rem The dll name being kddockwidgets-qt62.dll or kddockwidgets-qt6.dll respectively
rem Always use CMAKE_CONFIGURATION_TYPES to give consistent naming
set KDDOCKWIDGETSBUILDSPEC=-DCMAKE_CONFIGURATION_TYPES=Release -DCMAKE_CROSS_CONFIGS=all -DCMAKE_DEFAULT_BUILD_TYPE=Release -DCMAKE_DEFAULT_CONFIGS=Release -G "Ninja Multi-Config"
)
echo "Building KDDockWidgets..."
rmdir /S /Q "KDDockWidgets-%KDDOCKWIDGETS%"
%SEVENZIP% x "KDDockWidgets-%KDDOCKWIDGETS%.zip" || goto error
cd "KDDockWidgets-%KDDOCKWIDGETS%" || goto error
%PATCH% -p1 < "%SCRIPTDIR%\..\common\kddockwidgets-dodgy-include.patch" || goto error
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH="%INSTALLDIR%" -DCMAKE_INSTALL_PREFIX="%INSTALLDIR%" -DKDDockWidgets_QT6=true -DKDDockWidgets_EXAMPLES=false -DKDDockWidgets_FRONTENDS=qtwidgets -B build -G Ninja || goto error
cmake -B build -DCMAKE_PREFIX_PATH="%INSTALLDIR%" -DCMAKE_INSTALL_PREFIX="%INSTALLDIR%" -DKDDockWidgets_QT6=true -DKDDockWidgets_EXAMPLES=false -DKDDockWidgets_FRONTENDS=qtwidgets %KDDOCKWIDGETSBUILDSPEC% || goto error
cmake --build build --parallel || goto error
ninja -C build install || goto error
cd .. || goto error

View File

@@ -1,13 +0,0 @@
diff --git a/src/plugins/styles/modernwindows/qwindowsvistastyle.cpp b/src/plugins/styles/modernwindows/qwindowsvistastyle.cpp
index 208420d7e8..26ef6f31ef 100644
--- a/src/plugins/styles/modernwindows/qwindowsvistastyle.cpp
+++ b/src/plugins/styles/modernwindows/qwindowsvistastyle.cpp
@@ -4232,8 +4232,6 @@ QRect QWindowsVistaStyle::subElementRect(SubElement element, const QStyleOption
case SE_ItemViewItemDecoration:
rect = QWindowsStyle::subElementRect(element, option, widget);
- if (qstyleoption_cast<const QStyleOptionViewItem *>(option))
- rect.adjust(-2, 0, 2, 0);
break;
case SE_ItemViewItemFocusRect:

View File

@@ -19,7 +19,7 @@ jobs:
pr-message: |-
## Thank you for submitting a contribution to PCSX2
As this is your first pull request, [please be aware of the contributing guidelines](https://github.com/PCSX2/pcsx2/blob/master/.github/CONTRIBUTING.md).
As this is your first pull request, [please be aware of the contributing guidelines](https://pcsx2.net/docs/contributing/).
Additionally, as per recent changes in GitHub Actions, your pull request will need to be approved by a maintainer before GitHub Actions can run against it. [You can find more information about this change here.](https://github.blog/2021-04-22-github-actions-update-helping-maintainers-combat-bad-actors/)

View File

@@ -168,7 +168,7 @@ jobs:
!./bin/**/*.lib
- name: Install the Breakpad Symbol Generator
uses: baptiste0928/cargo-install@91c5da15570085bcde6f4d7aed98cb82d6769fd3
uses: baptiste0928/cargo-install@e38323ef017552d7f7af73a3f4db467f278310ed
with:
crate: dump_syms

View File

@@ -1,3 +1,34 @@
# 3.0.1 (2025-05-08)
- Fixed macros `D3D12MA_RECOMMENDED_ALLOCATOR_FLAGS`, `D3D12MA_RECOMMENDED_POOL_FLAGS` (#73).
# 3.0.0 (2025-05-05)
It has been a long time since the previous official release, so hopefully everyone has been using the latest code from "master" branch, which is always maintained in a good state, not the old version. For completeness, here is the list of changes since v2.0.1. The major version number has changed, so there are some compatibility-breaking changes, but the basic API stays the same and is mostly backward-compatible.
- Added helper structs: `CALLOCATION_DESC`, `CPOOL_DESC`, `CVIRTUAL_BLOCK_DESC`, `CVIRTUAL_ALLOCATION_DESC`.
- Added macros: `D3D12MA_RECOMMENDED_ALLOCATOR_FLAGS`, `D3D12MA_RECOMMENDED_HEAP_FLAGS`, `D3D12MA_RECOMMENDED_POOL_FLAGS`.
- Added functions: `Allocator::CreateResource3`, `CreateAliasingResource2`.
- They support parameters: `D3D12_BARRIER_LAYOUT InitialLayout`, `const DXGI_FORMAT* pCastableFormats`.
- They require recent DirectX 12 Agility SDK. To use them, `ID3D12Device10` must be available.
To use non-empty list of castable formats, `ID3D12Device12` must be available.
- Added support for GPU Upload Heaps (`D3D12_HEAP_TYPE_GPU_UPLOAD`).
- Requires recent DirectX 12 Agility SDK. Support on the user's machine is available only when supported by the motherboard, GPU, drivers, and enabled as "Resizable BAR" in UEFI settings. It can be queried using new `Allocator::IsGPUUploadHeapSupported` function.
- `TotalStatistics::HeapType` array was extended from 4 to 5 elements.
- Added missing function `Allocator::CreateAliasingResource1`.
- Added `POOL_DESC::ResidencyPriority` member.
- Removed `Allocation::WasZeroInitialized` function. It wasn't fully implemented anyway.
- Added `POOL_FLAG_ALWAYS_COMMITTED`.
- Added a heuristic that prefers creating small buffers as committed to save memory.
- It is enabled by default. It can be disabled by new flag `ALLOCATOR_FLAG_DONT_PREFER_SMALL_BUFFERS_COMMITTED`.
- Macro `D3D12MA_OPTIONS16_SUPPORTED` is no longer exposed in the header or Cmake script.
It is defined automatically based on the Agility SDK version.
- Added macro `D3D12MA_DEBUG_LOG`, which can be used to log unfreed allocations.
- Many improvements in the documentation, including new chapters: "Frequently asked questions", "Optimal resource allocation".
- Countless fixes and improvements, including performance optimizations, compatibility with various compilers, tests.
- Major changes in the Cmake script.
- Fixes in "GpuMemDumpVis.py" script.
# 2.0.1 (2022-04-05)
A maintenance release with some bug fixes and improvements. There are no changes in the library API.

View File

@@ -1,4 +1,4 @@
Copyright (c) 2019-2024 Advanced Micro Devices, Inc. All rights reserved.
Copyright (c) 2019-2025 Advanced Micro Devices, Inc. All rights reserved.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

View File

@@ -10,10 +10,6 @@ Easy to integrate memory allocation library for Direct3D 12.
**Product page:** [D3D12 Memory Allocator on GPUOpen](https://gpuopen.com/gaming-product/d3d12-memory-allocator/)
**Build status:**
Windows: [![Build status](https://ci.appveyor.com/api/projects/status/860i07bxv55ydgvg?svg=true)](https://ci.appveyor.com/project/adam-sawicki-amd/d3d12memoryallocator)
[![Average time to resolve an issue](http://isitmaintained.com/badge/resolution/GPUOpen-LibrariesAndSDKs/D3D12MemoryAllocator.svg)](http://isitmaintained.com/project/GPUOpen-LibrariesAndSDKs/D3D12MemoryAllocator "Average time to resolve an issue")
# Problem
@@ -91,7 +87,7 @@ With this one function call:
# Binaries
The release comes with precompiled binary executable for "D3D12Sample" application which contains test suite. It is compiled using Visual Studio 2019, so it requires appropriate libraries to work, including "MSVCP140.dll", "VCRUNTIME140.dll", "VCRUNTIME140_1.dll". If its launch fails with error message telling about those files missing, please download and install [Microsoft Visual C++ Redistributable for Visual Studio 2015, 2017 and 2019](https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads), "x64" version.
The release comes with precompiled binary executable for "D3D12Sample" application which contains test suite. It is compiled using Visual Studio 2022, so it requires appropriate libraries to work, including "MSVCP140.dll", "VCRUNTIME140.dll", "VCRUNTIME140_1.dll". If its launch fails with error message telling about those files missing, please download and install [Microsoft Visual C++ Redistributable](https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170), "X64" version.
# Copyright notice
@@ -113,7 +109,8 @@ For more information see [NOTICES.txt](NOTICES.txt).
- **[Qt Project](https://github.com/qt)**
- **[Ghost of Tsushima: Director's Cut PC](https://www.youtube.com/watch?v=cPKBDbCYctc&t=698s)** - Information avaliable in 11:38 of credits
- **[Godot Engine](https://github.com/godotengine/godot/)** - multi-platform 2D and 3D game engine. License: MIT.
- **[The Forge](https://github.com/ConfettiFX/The-Forge)** - cross-platform rendering framework. Apache License 2.0.
- **[Wicked Engine<img src="https://github.com/turanszkij/WickedEngine/blob/master/Content/logo_small.png" width="28px" align="center"/>](https://github.com/turanszkij/WickedEngine)** - 3D engine with modern graphics
- **[Wicked Engine](https://github.com/turanszkij/WickedEngine)** - 3D engine with modern graphics
[Some other projects on GitHub](https://github.com/search?q=D3D12MemAlloc.h&type=Code) and some game development studios that use DX12 in their games.

File diff suppressed because it is too large Load Diff

View File

@@ -1,5 +1,5 @@
//
// Copyright (c) 2019-2024 Advanced Micro Devices, Inc. All rights reserved.
// Copyright (c) 2019-2025 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
@@ -33,6 +33,13 @@
#include <shared_mutex>
#endif
// Includes needed for MinGW - see #71.
#ifndef _MSC_VER
#include <guiddef.h>
// guiddef.h must be included first.
#include <dxguids.h>
#endif
////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////
//
@@ -42,6 +49,14 @@
////////////////////////////////////////////////////////////////////////////////
#ifndef _D3D12MA_CONFIGURATION
#if !defined(D3D12MA_CPP20)
#if __cplusplus >= 202002L || _MSVC_LANG >= 202002L // C++20
#define D3D12MA_CPP20 1
#else
#define D3D12MA_CPP20 0
#endif
#endif
#ifdef _WIN32
#if !defined(WINVER) || WINVER < 0x0600
#error Required at least WinAPI version supporting: client = Windows Vista, server = Windows Server 2008.
@@ -64,6 +79,10 @@
#define D3D12MA_ASSERT(cond) assert(cond)
#endif
#if D3D12MA_CPP20
#include <bit>
#endif
// Assert that will be called very often, like inside data structures e.g. operator[].
// Making it non-empty can make program slow.
#ifndef D3D12MA_HEAVY_ASSERT
@@ -107,6 +126,14 @@ especially to test compatibility with D3D12_RESOURCE_HEAP_TIER_1 on modern GPUs.
#define D3D12MA_DEFAULT_BLOCK_SIZE (64ull * 1024 * 1024)
#endif
#ifndef D3D12MA_OPTIONS16_SUPPORTED
#if D3D12_SDK_VERSION >= 610
#define D3D12MA_OPTIONS16_SUPPORTED 1
#else
#define D3D12MA_OPTIONS16_SUPPORTED 0
#endif
#endif
#ifndef D3D12MA_DEBUG_LOG
#define D3D12MA_DEBUG_LOG(format, ...)
/*
@@ -128,10 +155,6 @@ especially to test compatibility with D3D12_RESOURCE_HEAP_TIER_1 on modern GPUs.
#define D3D12MA_IID_PPV_ARGS(ppType) __uuidof(**(ppType)), reinterpret_cast<void**>(ppType)
#ifdef __ID3D12Device8_INTERFACE_DEFINED__
#define D3D12MA_CREATE_NOT_ZEROED_AVAILABLE 1
#endif
namespace D3D12MA
{
static constexpr UINT HEAP_TYPE_COUNT = 5;
@@ -292,6 +315,10 @@ static UINT8 BitScanLSB(UINT64 mask)
if (_BitScanForward64(&pos, mask))
return static_cast<UINT8>(pos);
return UINT8_MAX;
#elif D3D12MA_CPP20
if (mask != 0)
return static_cast<uint8_t>(std::countr_zero(mask));
return UINT8_MAX;
#elif defined __GNUC__ || defined __clang__
return static_cast<UINT8>(__builtin_ffsll(mask)) - 1U;
#else
@@ -314,6 +341,10 @@ static UINT8 BitScanLSB(UINT32 mask)
if (_BitScanForward(&pos, mask))
return static_cast<UINT8>(pos);
return UINT8_MAX;
#elif D3D12MA_CPP20
if (mask != 0)
return static_cast<uint8_t>(std::countr_zero(mask));
return UINT8_MAX;
#elif defined __GNUC__ || defined __clang__
return static_cast<UINT8>(__builtin_ffs(mask)) - 1U;
#else
@@ -336,6 +367,9 @@ static UINT8 BitScanMSB(UINT64 mask)
unsigned long pos;
if (_BitScanReverse64(&pos, mask))
return static_cast<UINT8>(pos);
#elif D3D12MA_CPP20
if (mask != 0)
return 63 - static_cast<uint8_t>(std::countl_zero(mask));
#elif defined __GNUC__ || defined __clang__
if (mask)
return 63 - static_cast<UINT8>(__builtin_clzll(mask));
@@ -358,6 +392,9 @@ static UINT8 BitScanMSB(UINT32 mask)
unsigned long pos;
if (_BitScanReverse(&pos, mask))
return static_cast<UINT8>(pos);
#elif D3D12MA_CPP20
if (mask != 0)
return 31 - static_cast<uint8_t>(std::countl_zero(mask));
#elif defined __GNUC__ || defined __clang__
if (mask)
return 31 - static_cast<UINT8>(__builtin_clz(mask));
@@ -2791,7 +2828,7 @@ class AllocationObjectAllocator
D3D12MA_CLASS_NO_COPY(AllocationObjectAllocator);
public:
AllocationObjectAllocator(const ALLOCATION_CALLBACKS& allocationCallbacks, bool useMutex)
: m_Allocator(allocationCallbacks, 1024), m_UseMutex(useMutex) {}
: m_UseMutex(useMutex), m_Allocator(allocationCallbacks, 1024) {}
template<typename... Types>
Allocation* Allocate(Types... args);
@@ -2982,7 +3019,7 @@ void BlockMetadata::DebugLogAllocation(UINT64 offset, UINT64 size, void* private
LPCWSTR name = allocation->GetName();
D3D12MA_DEBUG_LOG(L"UNFREED ALLOCATION; Offset: %llu; Size: %llu; PrivateData: %p; Name: %s",
offset, size, privateData, name ? name : L"D3D12MA_Empty");
offset, size, privateData, name ? name : L"");
}
}
@@ -5369,8 +5406,8 @@ struct CREATE_RESOURCE_PARAMS
{
CREATE_RESOURCE_PARAMS() = delete;
CREATE_RESOURCE_PARAMS(
const D3D12_RESOURCE_DESC* pResourceDesc,
D3D12_RESOURCE_STATES InitialResourceState,
const D3D12_RESOURCE_DESC* pResourceDesc,
D3D12_RESOURCE_STATES InitialResourceState,
const D3D12_CLEAR_VALUE* pOptimizedClearValue)
: Variant(VARIANT_WITH_STATE)
, pResourceDesc(pResourceDesc)
@@ -5380,8 +5417,8 @@ struct CREATE_RESOURCE_PARAMS
}
#ifdef __ID3D12Device8_INTERFACE_DEFINED__
CREATE_RESOURCE_PARAMS(
const D3D12_RESOURCE_DESC1* pResourceDesc,
D3D12_RESOURCE_STATES InitialResourceState,
const D3D12_RESOURCE_DESC1* pResourceDesc,
D3D12_RESOURCE_STATES InitialResourceState,
const D3D12_CLEAR_VALUE* pOptimizedClearValue)
: Variant(VARIANT_WITH_STATE_AND_DESC1)
, pResourceDesc1(pResourceDesc)
@@ -5396,7 +5433,7 @@ struct CREATE_RESOURCE_PARAMS
D3D12_BARRIER_LAYOUT InitialLayout,
const D3D12_CLEAR_VALUE* pOptimizedClearValue,
UINT32 NumCastableFormats,
DXGI_FORMAT* pCastableFormats)
const DXGI_FORMAT* pCastableFormats)
: Variant(VARIANT_WITH_LAYOUT)
, pResourceDesc1(pResourceDesc)
, InitialLayout(InitialLayout)
@@ -5466,7 +5503,7 @@ struct CREATE_RESOURCE_PARAMS
D3D12MA_ASSERT(Variant >= VARIANT_WITH_LAYOUT);
return NumCastableFormats;
}
DXGI_FORMAT* GetCastableFormats() const
const DXGI_FORMAT* GetCastableFormats() const
{
D3D12MA_ASSERT(Variant >= VARIANT_WITH_LAYOUT);
return pCastableFormats;
@@ -5491,7 +5528,7 @@ private:
const D3D12_CLEAR_VALUE* pOptimizedClearValue;
#ifdef __ID3D12Device10_INTERFACE_DEFINED__
UINT32 NumCastableFormats;
DXGI_FORMAT* pCastableFormats;
const DXGI_FORMAT* pCastableFormats;
#endif
};
@@ -5541,6 +5578,7 @@ public:
UINT64 size,
UINT64 alignment,
const ALLOCATION_DESC& allocDesc,
bool committedAllowed,
size_t allocationCount,
Allocation** pAllocations);
@@ -5551,6 +5589,7 @@ public:
UINT64 alignment,
const ALLOCATION_DESC& allocDesc,
const CREATE_RESOURCE_PARAMS& createParams,
bool committedAllowed,
Allocation** ppAllocation,
REFIID riidResource,
void** ppvResource);
@@ -5601,6 +5640,7 @@ private:
UINT64 size,
UINT64 alignment,
const ALLOCATION_DESC& allocDesc,
bool committedAllowed,
Allocation** pAllocation);
HRESULT AllocateFromBlock(
@@ -5706,29 +5746,31 @@ HRESULT CurrentBudgetData::UpdateBudget(IDXGIAdapter3* adapter3, bool useMutex)
DXGI_QUERY_VIDEO_MEMORY_INFO infoLocal = {};
DXGI_QUERY_VIDEO_MEMORY_INFO infoNonLocal = {};
const HRESULT hrLocal = adapter3->QueryVideoMemoryInfo(0, DXGI_MEMORY_SEGMENT_GROUP_LOCAL, &infoLocal);
if (FAILED(hrLocal))
{
return hrLocal;
}
const HRESULT hrNonLocal = adapter3->QueryVideoMemoryInfo(0, DXGI_MEMORY_SEGMENT_GROUP_NON_LOCAL, &infoNonLocal);
if (FAILED(hrNonLocal))
{
return hrNonLocal;
}
if (SUCCEEDED(hrLocal) || SUCCEEDED(hrNonLocal))
{
MutexLockWrite lockWrite(m_BudgetMutex, useMutex);
if (SUCCEEDED(hrLocal))
{
m_D3D12Usage[0] = infoLocal.CurrentUsage;
m_D3D12Budget[0] = infoLocal.Budget;
}
if (SUCCEEDED(hrNonLocal))
{
m_D3D12Usage[1] = infoNonLocal.CurrentUsage;
m_D3D12Budget[1] = infoNonLocal.Budget;
}
m_D3D12Usage[0] = infoLocal.CurrentUsage;
m_D3D12Budget[0] = infoLocal.Budget;
m_D3D12Usage[1] = infoNonLocal.CurrentUsage;
m_D3D12Budget[1] = infoNonLocal.Budget;
m_BlockBytesAtD3D12Fetch[0] = m_BlockBytes[0];
m_BlockBytesAtD3D12Fetch[1] = m_BlockBytes[1];
m_OperationsSinceBudgetFetch = 0;
}
return FAILED(hrLocal) ? hrLocal : hrNonLocal;
return S_OK;
}
#endif // #if D3D12MA_DXGI_1_4
@@ -5847,6 +5889,7 @@ public:
AllocatorPimpl* GetAllocator() const { return m_Allocator; }
const POOL_DESC& GetDesc() const { return m_Desc; }
bool AlwaysCommitted() const { return (m_Desc.Flags & POOL_FLAG_ALWAYS_COMMITTED) != 0; }
bool SupportsCommittedAllocations() const { return m_Desc.BlockSize == 0; }
LPCWSTR GetName() const { return m_Name; }
@@ -5903,6 +5946,12 @@ public:
#endif
#ifdef __ID3D12Device8_INTERFACE_DEFINED__
ID3D12Device8* GetDevice8() const { return m_Device8; }
#endif
#ifdef __ID3D12Device10_INTERFACE_DEFINED__
ID3D12Device10* GetDevice10() const { return m_Device10; }
#endif
#ifdef __ID3D12Device12_INTERFACE_DEFINED__
ID3D12Device12* GetDevice12() const { return m_Device12; }
#endif
// Shortcut for "Allocation Callbacks", because this function is called so often.
const ALLOCATION_CALLBACKS& GetAllocs() const { return m_AllocationCallbacks; }
@@ -6011,6 +6060,9 @@ private:
#endif
#ifdef __ID3D12Device10_INTERFACE_DEFINED__
ID3D12Device10* m_Device10 = NULL; // AddRef, optional
#endif
#ifdef __ID3D12Device12_INTERFACE_DEFINED__
ID3D12Device12* m_Device12 = NULL; // AddRef, optional
#endif
IDXGIAdapter* m_Adapter; // AddRef
#if D3D12MA_DXGI_1_4
@@ -6071,12 +6123,26 @@ private:
HRESULT UpdateD3D12Budget();
D3D12_RESOURCE_ALLOCATION_INFO GetResourceAllocationInfoNative(const D3D12_RESOURCE_DESC& resourceDesc) const;
HRESULT GetResourceAllocationInfoMiddle(D3D12_RESOURCE_DESC& inOutResourceDesc,
UINT32 NumCastableFormats, const DXGI_FORMAT* pCastableFormats,
D3D12_RESOURCE_ALLOCATION_INFO& outAllocInfo) const;
#ifdef __ID3D12Device8_INTERFACE_DEFINED__
D3D12_RESOURCE_ALLOCATION_INFO GetResourceAllocationInfoNative(const D3D12_RESOURCE_DESC1& resourceDesc) const;
D3D12_RESOURCE_ALLOCATION_INFO GetResourceAllocationInfo2Native(const D3D12_RESOURCE_DESC1& resourceDesc) const;
HRESULT GetResourceAllocationInfoMiddle(D3D12_RESOURCE_DESC1& inOutResourceDesc,
UINT32 NumCastableFormats, const DXGI_FORMAT* pCastableFormats,
D3D12_RESOURCE_ALLOCATION_INFO& outAllocInfo) const;
#endif
#ifdef __ID3D12Device12_INTERFACE_DEFINED__
D3D12_RESOURCE_ALLOCATION_INFO GetResourceAllocationInfo3Native(const D3D12_RESOURCE_DESC1& resourceDesc,
UINT32 NumCastableFormats, const DXGI_FORMAT* pCastableFormats) const;
#endif
template<typename D3D12_RESOURCE_DESC_T>
D3D12_RESOURCE_ALLOCATION_INFO GetResourceAllocationInfo(D3D12_RESOURCE_DESC_T& inOutResourceDesc) const;
HRESULT GetResourceAllocationInfo(D3D12_RESOURCE_DESC_T& inOutResourceDesc,
UINT32 NumCastableFormats, const DXGI_FORMAT* pCastableFormats,
D3D12_RESOURCE_ALLOCATION_INFO& outAllocInfo) const;
bool NewAllocationWithinBudget(D3D12_HEAP_TYPE heapType, UINT64 size);
@@ -6148,6 +6214,10 @@ HRESULT AllocatorPimpl::Init(const ALLOCATOR_DESC& desc)
m_Device->QueryInterface(D3D12MA_IID_PPV_ARGS(&m_Device10));
#endif
#ifdef __ID3D12Device12_INTERFACE_DEFINED__
m_Device->QueryInterface(D3D12MA_IID_PPV_ARGS(&m_Device12));
#endif
HRESULT hr = m_Adapter->GetDesc(&m_AdapterDesc);
if (FAILED(hr))
{
@@ -6163,9 +6233,6 @@ HRESULT AllocatorPimpl::Init(const ALLOCATOR_DESC& desc)
m_D3D12Options.ResourceHeapTier = (D3D12MA_FORCE_RESOURCE_HEAP_TIER);
#endif
// You must define this macro to like `#define D3D12MA_OPTIONS16_SUPPORTED 1` to enable GPU Upload Heaps!
// Unfortunately there is no way to programmatically check if the included <d3d12.h> defines D3D12_FEATURE_DATA_D3D12_OPTIONS16 or not.
// Main interfaces have respective macros like __ID3D12Device4_INTERFACE_DEFINED__, but structures like this do not.
#if D3D12MA_OPTIONS16_SUPPORTED
{
D3D12_FEATURE_DATA_D3D12_OPTIONS16 options16 = {};
@@ -6175,7 +6242,7 @@ HRESULT AllocatorPimpl::Init(const ALLOCATOR_DESC& desc)
m_GPUUploadHeapSupported = options16.GPUUploadHeapSupported;
}
}
#endif
#endif // #if D3D12MA_OPTIONS16_SUPPORTED
hr = m_Device->CheckFeatureSupport(D3D12_FEATURE_ARCHITECTURE, &m_D3D12Architecture, sizeof(m_D3D12Architecture));
if (FAILED(hr))
@@ -6223,6 +6290,9 @@ HRESULT AllocatorPimpl::Init(const ALLOCATOR_DESC& desc)
AllocatorPimpl::~AllocatorPimpl()
{
#ifdef __ID3D12Device12_INTERFACE_DEFINED__
SAFE_RELEASE(m_Device12);
#endif
#ifdef __ID3D12Device10_INTERFACE_DEFINED__
SAFE_RELEASE(m_Device10);
#endif
@@ -6319,12 +6389,15 @@ HRESULT AllocatorPimpl::CreatePlacedResourceWrap(
{
return E_NOINTERFACE;
}
// Microsoft defined pCastableFormats parameter as pointer to non-const and only fixed it in later Agility SDK,
// thus we need const_cast.
return m_Device10->CreatePlacedResource2(pHeap, HeapOffset,
createParams.GetResourceDesc1(), createParams.GetInitialLayout(),
createParams.GetOptimizedClearValue(), createParams.GetNumCastableFormats(),
createParams.GetCastableFormats(), riidResource, ppvResource);
} else
const_cast<DXGI_FORMAT*>(createParams.GetCastableFormats()), riidResource, ppvResource);
}
#endif
#ifdef __ID3D12Device8_INTERFACE_DEFINED__
if (createParams.Variant == CREATE_RESOURCE_PARAMS::VARIANT_WITH_STATE_AND_DESC1)
{
@@ -6335,21 +6408,19 @@ HRESULT AllocatorPimpl::CreatePlacedResourceWrap(
return m_Device8->CreatePlacedResource1(pHeap, HeapOffset,
createParams.GetResourceDesc1(), createParams.GetInitialResourceState(),
createParams.GetOptimizedClearValue(), riidResource, ppvResource);
} else
}
#endif
if (createParams.Variant == CREATE_RESOURCE_PARAMS::VARIANT_WITH_STATE)
{
return m_Device->CreatePlacedResource(pHeap, HeapOffset,
createParams.GetResourceDesc(), createParams.GetInitialResourceState(),
createParams.GetOptimizedClearValue(), riidResource, ppvResource);
}
else
{
D3D12MA_ASSERT(0);
return E_INVALIDARG;
}
}
D3D12MA_ASSERT(0);
return E_INVALIDARG;
}
HRESULT AllocatorPimpl::CreateResource(
const ALLOCATION_DESC* pAllocDesc,
@@ -6366,6 +6437,7 @@ HRESULT AllocatorPimpl::CreateResource(
*ppvResource = NULL;
}
HRESULT hr = E_NOINTERFACE;
CREATE_RESOURCE_PARAMS finalCreateParams = createParams;
D3D12_RESOURCE_DESC finalResourceDesc;
#ifdef __ID3D12Device8_INTERFACE_DEFINED__
@@ -6376,45 +6448,49 @@ HRESULT AllocatorPimpl::CreateResource(
{
finalResourceDesc = *createParams.GetResourceDesc();
finalCreateParams.AccessResourceDesc() = &finalResourceDesc;
resAllocInfo = GetResourceAllocationInfo(finalResourceDesc);
hr = GetResourceAllocationInfo(finalResourceDesc, 0, NULL, resAllocInfo);
}
#ifdef __ID3D12Device8_INTERFACE_DEFINED__
else if (createParams.Variant == CREATE_RESOURCE_PARAMS::VARIANT_WITH_STATE_AND_DESC1)
{
if (!m_Device8)
if (m_Device8 != NULL)
{
return E_NOINTERFACE;
finalResourceDesc1 = *createParams.GetResourceDesc1();
finalCreateParams.AccessResourceDesc1() = &finalResourceDesc1;
hr = GetResourceAllocationInfo(finalResourceDesc1, 0, NULL, resAllocInfo);
}
finalResourceDesc1 = *createParams.GetResourceDesc1();
finalCreateParams.AccessResourceDesc1() = &finalResourceDesc1;
resAllocInfo = GetResourceAllocationInfo(finalResourceDesc1);
}
#endif
#ifdef __ID3D12Device10_INTERFACE_DEFINED__
else if (createParams.Variant == CREATE_RESOURCE_PARAMS::VARIANT_WITH_LAYOUT)
{
if (!m_Device10)
if (m_Device10 != NULL)
{
return E_NOINTERFACE;
finalResourceDesc1 = *createParams.GetResourceDesc1();
finalCreateParams.AccessResourceDesc1() = &finalResourceDesc1;
hr = GetResourceAllocationInfo(finalResourceDesc1,
createParams.GetNumCastableFormats(), createParams.GetCastableFormats(), resAllocInfo);
}
finalResourceDesc1 = *createParams.GetResourceDesc1();
finalCreateParams.AccessResourceDesc1() = &finalResourceDesc1;
resAllocInfo = GetResourceAllocationInfo(finalResourceDesc1);
}
#endif
else
{
D3D12MA_ASSERT(0);
return E_INVALIDARG;
hr = E_INVALIDARG;
}
if (FAILED(hr))
return hr;
D3D12MA_ASSERT(IsPow2(resAllocInfo.Alignment));
// We've seen UINT64_MAX returned when the call to GetResourceAllocationInfo was invalid.
D3D12MA_ASSERT(resAllocInfo.SizeInBytes != UINT64_MAX);
D3D12MA_ASSERT(resAllocInfo.SizeInBytes > 0);
BlockVector* blockVector = NULL;
CommittedAllocationParameters committedAllocationParams = {};
bool preferCommitted = false;
HRESULT hr;
#ifdef __ID3D12Device8_INTERFACE_DEFINED__
if (createParams.Variant >= CREATE_RESOURCE_PARAMS::VARIANT_WITH_STATE_AND_DESC1)
{
@@ -6445,7 +6521,7 @@ HRESULT AllocatorPimpl::CreateResource(
if (blockVector != NULL)
{
hr = blockVector->CreateResource(resAllocInfo.SizeInBytes, resAllocInfo.Alignment,
*pAllocDesc, finalCreateParams,
*pAllocDesc, finalCreateParams, committedAllocationParams.IsValid(),
ppAllocation, riidResource, ppvResource);
if (SUCCEEDED(hr))
return hr;
@@ -6488,7 +6564,7 @@ HRESULT AllocatorPimpl::AllocateMemory(
if (blockVector != NULL)
{
hr = blockVector->Allocate(pAllocInfo->SizeInBytes, pAllocInfo->Alignment,
*pAllocDesc, 1, (Allocation**)ppAllocation);
*pAllocDesc, committedAllocationParams.IsValid(), 1, (Allocation**)ppAllocation);
if (SUCCEEDED(hr))
return hr;
}
@@ -6510,6 +6586,7 @@ HRESULT AllocatorPimpl::CreateAliasingResource(
{
*ppvResource = NULL;
HRESULT hr = E_NOINTERFACE;
CREATE_RESOURCE_PARAMS finalCreateParams = createParams;
D3D12_RESOURCE_DESC finalResourceDesc;
#ifdef __ID3D12Device8_INTERFACE_DEFINED__
@@ -6520,37 +6597,40 @@ HRESULT AllocatorPimpl::CreateAliasingResource(
{
finalResourceDesc = *createParams.GetResourceDesc();
finalCreateParams.AccessResourceDesc() = &finalResourceDesc;
resAllocInfo = GetResourceAllocationInfo(finalResourceDesc);
hr = GetResourceAllocationInfo(finalResourceDesc, 0, NULL, resAllocInfo);
}
#ifdef __ID3D12Device8_INTERFACE_DEFINED__
else if (createParams.Variant == CREATE_RESOURCE_PARAMS::VARIANT_WITH_STATE_AND_DESC1)
{
if (!m_Device8)
if (m_Device8 != NULL)
{
return E_NOINTERFACE;
finalResourceDesc1 = *createParams.GetResourceDesc1();
finalCreateParams.AccessResourceDesc1() = &finalResourceDesc1;
hr = GetResourceAllocationInfo(finalResourceDesc1, 0, NULL, resAllocInfo);
}
finalResourceDesc1 = *createParams.GetResourceDesc1();
finalCreateParams.AccessResourceDesc1() = &finalResourceDesc1;
resAllocInfo = GetResourceAllocationInfo(finalResourceDesc1);
}
#endif
#ifdef __ID3D12Device10_INTERFACE_DEFINED__
else if (createParams.Variant == CREATE_RESOURCE_PARAMS::VARIANT_WITH_LAYOUT)
{
if (!m_Device10)
if (m_Device10 != NULL)
{
return E_NOINTERFACE;
finalResourceDesc1 = *createParams.GetResourceDesc1();
finalCreateParams.AccessResourceDesc1() = &finalResourceDesc1;
hr = GetResourceAllocationInfo(finalResourceDesc1,
createParams.GetNumCastableFormats(), createParams.GetCastableFormats(), resAllocInfo);
}
finalResourceDesc1 = *createParams.GetResourceDesc1();
finalCreateParams.AccessResourceDesc1() = &finalResourceDesc1;
resAllocInfo = GetResourceAllocationInfo(finalResourceDesc1);
}
#endif
else
{
D3D12MA_ASSERT(0);
return E_INVALIDARG;
hr = E_INVALIDARG;
}
if (FAILED(hr))
return hr;
D3D12MA_ASSERT(IsPow2(resAllocInfo.Alignment));
D3D12MA_ASSERT(resAllocInfo.SizeInBytes > 0);
@@ -6773,42 +6853,41 @@ void AllocatorPimpl::GetBudget(Budget* outLocalBudget, Budget* outNonLocalBudget
outLocalBudget ? &outLocalBudget->BudgetBytes : NULL,
outNonLocalBudget ? &outNonLocalBudget->UsageBytes : NULL,
outNonLocalBudget ? &outNonLocalBudget->BudgetBytes : NULL);
return;
}
else
if (SUCCEEDED(UpdateD3D12Budget()))
{
UpdateD3D12Budget();
GetBudget(outLocalBudget, outNonLocalBudget); // Recursion
GetBudget(outLocalBudget, outNonLocalBudget); // Recursion.
return;
}
}
else
#endif
// Fallback path - manual calculation, not real budget.
if (outLocalBudget)
{
if (outLocalBudget)
{
outLocalBudget->UsageBytes = outLocalBudget->Stats.BlockBytes;
outLocalBudget->BudgetBytes = GetMemoryCapacity(DXGI_MEMORY_SEGMENT_GROUP_LOCAL_COPY) * 8 / 10; // 80% heuristics.
}
if (outNonLocalBudget)
{
outNonLocalBudget->UsageBytes = outNonLocalBudget->Stats.BlockBytes;
outNonLocalBudget->BudgetBytes = GetMemoryCapacity(DXGI_MEMORY_SEGMENT_GROUP_NON_LOCAL_COPY) * 8 / 10; // 80% heuristics.
}
outLocalBudget->UsageBytes = outLocalBudget->Stats.BlockBytes;
outLocalBudget->BudgetBytes = GetMemoryCapacity(DXGI_MEMORY_SEGMENT_GROUP_LOCAL_COPY) * 8 / 10; // 80% heuristics.
}
if (outNonLocalBudget)
{
outNonLocalBudget->UsageBytes = outNonLocalBudget->Stats.BlockBytes;
outNonLocalBudget->BudgetBytes = GetMemoryCapacity(DXGI_MEMORY_SEGMENT_GROUP_NON_LOCAL_COPY) * 8 / 10; // 80% heuristics.
}
}
void AllocatorPimpl::GetBudgetForHeapType(Budget& outBudget, D3D12_HEAP_TYPE heapType)
{
switch (heapType)
const bool isLocal = StandardHeapTypeToMemorySegmentGroup(heapType) ==
DXGI_MEMORY_SEGMENT_GROUP_LOCAL_COPY;
if (isLocal)
{
case D3D12_HEAP_TYPE_DEFAULT:
case D3D12_HEAP_TYPE_GPU_UPLOAD_COPY:
GetBudget(&outBudget, NULL);
break;
case D3D12_HEAP_TYPE_UPLOAD:
case D3D12_HEAP_TYPE_READBACK:
}
else
{
GetBudget(NULL, &outBudget);
break;
default: D3D12MA_ASSERT(0);
}
}
@@ -7248,12 +7327,15 @@ HRESULT AllocatorPimpl::AllocateCommittedResource(
{
return E_NOINTERFACE;
}
// Microsoft defined pCastableFormats parameter as pointer to non-const and only fixed it in later Agility SDK,
// thus we need const_cast.
hr = m_Device10->CreateCommittedResource3(
&committedAllocParams.m_HeapProperties,
committedAllocParams.m_HeapFlags & ~RESOURCE_CLASS_HEAP_FLAGS,
createParams.GetResourceDesc1(), createParams.GetInitialLayout(),
createParams.GetOptimizedClearValue(), committedAllocParams.m_ProtectedSession,
createParams.GetNumCastableFormats(), createParams.GetCastableFormats(),
createParams.GetNumCastableFormats(), const_cast<DXGI_FORMAT*>(createParams.GetCastableFormats()),
D3D12MA_IID_PPV_ARGS(&res));
} else
#endif
@@ -7396,8 +7478,8 @@ HRESULT AllocatorPimpl::CalcAllocationParams(const ALLOCATION_DESC& allocDesc, U
outCommittedAllocationParams = CommittedAllocationParameters();
outPreferCommitted = false;
D3D12MA_ASSERT((allocDesc.HeapType != D3D12_HEAP_TYPE_GPU_UPLOAD_COPY || IsGPUUploadHeapSupported()) &&
"Trying to allocate from D3D12_HEAP_TYPE_GPU_UPLOAD while GPUUploadHeapSupported == FALSE or D3D12MA_OPTIONS16_SUPPORTED macro was not defined when compiling D3D12MA library.");
if (allocDesc.HeapType == D3D12_HEAP_TYPE_GPU_UPLOAD_COPY && !IsGPUUploadHeapSupported())
return E_NOTIMPL;
bool msaaAlwaysCommitted;
if (allocDesc.CustomPool != NULL)
@@ -7405,7 +7487,8 @@ HRESULT AllocatorPimpl::CalcAllocationParams(const ALLOCATION_DESC& allocDesc, U
PoolPimpl* const pool = allocDesc.CustomPool->m_Pimpl;
msaaAlwaysCommitted = pool->GetBlockVector()->DeniesMsaaTextures();
outBlockVector = pool->GetBlockVector();
if(!pool->AlwaysCommitted())
outBlockVector = pool->GetBlockVector();
const auto& desc = pool->GetDesc();
outCommittedAllocationParams.m_ProtectedSession = desc.pProtectedSession;
@@ -7444,12 +7527,6 @@ HRESULT AllocatorPimpl::CalcAllocationParams(const ALLOCATION_DESC& allocDesc, U
outPreferCommitted = true;
}
}
const D3D12_HEAP_FLAGS extraHeapFlags = allocDesc.ExtraHeapFlags & ~RESOURCE_CLASS_HEAP_FLAGS;
if (outBlockVector != NULL && extraHeapFlags != 0)
{
outBlockVector = NULL;
}
}
if ((allocDesc.Flags & ALLOCATION_FLAG_COMMITTED) != 0 ||
@@ -7479,12 +7556,7 @@ UINT AllocatorPimpl::CalcDefaultPoolIndex(const ALLOCATION_DESC& allocDesc, Reso
D3D12_HEAP_FLAGS extraHeapFlags = allocDesc.ExtraHeapFlags & ~RESOURCE_CLASS_HEAP_FLAGS;
#if D3D12MA_CREATE_NOT_ZEROED_AVAILABLE
// If allocator was created with ALLOCATOR_FLAG_DEFAULT_POOLS_NOT_ZEROED, also ignore
// D3D12_HEAP_FLAG_CREATE_NOT_ZEROED.
if(m_DefaultPoolsNotZeroed)
{
extraHeapFlags &= ~D3D12_HEAP_FLAG_CREATE_NOT_ZEROED;
}
extraHeapFlags &= ~D3D12_HEAP_FLAG_CREATE_NOT_ZEROED;
#endif
if (extraHeapFlags != 0)
@@ -7603,7 +7675,7 @@ D3D12_RESOURCE_ALLOCATION_INFO AllocatorPimpl::GetResourceAllocationInfoNative(c
}
#ifdef __ID3D12Device8_INTERFACE_DEFINED__
D3D12_RESOURCE_ALLOCATION_INFO AllocatorPimpl::GetResourceAllocationInfoNative(const D3D12_RESOURCE_DESC1& resourceDesc) const
D3D12_RESOURCE_ALLOCATION_INFO AllocatorPimpl::GetResourceAllocationInfo2Native(const D3D12_RESOURCE_DESC1& resourceDesc) const
{
D3D12MA_ASSERT(m_Device8 != NULL);
D3D12_RESOURCE_ALLOCATION_INFO1 info1Unused;
@@ -7619,8 +7691,71 @@ D3D12_RESOURCE_ALLOCATION_INFO AllocatorPimpl::GetResourceAllocationInfoNative(c
}
#endif // #ifdef __ID3D12Device8_INTERFACE_DEFINED__
#ifdef __ID3D12Device12_INTERFACE_DEFINED__
D3D12_RESOURCE_ALLOCATION_INFO AllocatorPimpl::GetResourceAllocationInfo3Native(const D3D12_RESOURCE_DESC1& resourceDesc,
UINT32 NumCastableFormats, const DXGI_FORMAT* pCastableFormats) const
{
D3D12MA_ASSERT(m_Device12 != NULL);
D3D12_RESOURCE_ALLOCATION_INFO1 info1Unused;
// This is how new D3D12 headers define GetResourceAllocationInfo function -
// different signature depending on these macros.
#if defined(_MSC_VER) || !defined(_WIN32)
return m_Device12->GetResourceAllocationInfo3(0, 1, &resourceDesc,
&NumCastableFormats, &pCastableFormats, &info1Unused);
#else
D3D12_RESOURCE_ALLOCATION_INFO retVal;
return *m_Device12->GetResourceAllocationInfo3(&retVal, 0, 1, &resourceDesc,
&NumCastableFormats, &pCastableFormats, &info1Unused);
#endif
}
#endif // #ifdef __ID3D12Device12_INTERFACE_DEFINED__
HRESULT AllocatorPimpl::GetResourceAllocationInfoMiddle(
D3D12_RESOURCE_DESC& inOutResourceDesc,
UINT32 NumCastableFormats, const DXGI_FORMAT* pCastableFormats,
D3D12_RESOURCE_ALLOCATION_INFO& outAllocInfo) const
{
if (NumCastableFormats > 0)
{
return E_NOTIMPL;
}
outAllocInfo = GetResourceAllocationInfoNative(inOutResourceDesc);
return outAllocInfo.SizeInBytes != UINT64_MAX ? S_OK : E_INVALIDARG;
}
#ifdef __ID3D12Device8_INTERFACE_DEFINED__
HRESULT AllocatorPimpl::GetResourceAllocationInfoMiddle(
D3D12_RESOURCE_DESC1& inOutResourceDesc,
UINT32 NumCastableFormats, const DXGI_FORMAT* pCastableFormats,
D3D12_RESOURCE_ALLOCATION_INFO& outAllocInfo) const
{
if (NumCastableFormats > 0)
{
#ifdef __ID3D12Device12_INTERFACE_DEFINED__
if (m_Device12 != NULL)
{
outAllocInfo = GetResourceAllocationInfo3Native(inOutResourceDesc, NumCastableFormats, pCastableFormats);
return outAllocInfo.SizeInBytes != UINT64_MAX ? S_OK : E_INVALIDARG;
}
#else
return E_NOTIMPL;
#endif
}
outAllocInfo = GetResourceAllocationInfo2Native(inOutResourceDesc);
return outAllocInfo.SizeInBytes != UINT64_MAX ? S_OK : E_INVALIDARG;
}
#endif // #ifdef __ID3D12Device8_INTERFACE_DEFINED__
template<typename D3D12_RESOURCE_DESC_T>
D3D12_RESOURCE_ALLOCATION_INFO AllocatorPimpl::GetResourceAllocationInfo(D3D12_RESOURCE_DESC_T& inOutResourceDesc) const
HRESULT AllocatorPimpl::GetResourceAllocationInfo(
D3D12_RESOURCE_DESC_T& inOutResourceDesc,
UINT32 NumCastableFormats, const DXGI_FORMAT* pCastableFormats,
D3D12_RESOURCE_ALLOCATION_INFO& outAllocInfo) const
{
#ifdef __ID3D12Device1_INTERFACE_DEFINED__
/* Optional optimization: Microsoft documentation says:
@@ -7634,12 +7769,15 @@ D3D12_RESOURCE_ALLOCATION_INFO AllocatorPimpl::GetResourceAllocationInfo(D3D12_R
if (inOutResourceDesc.Alignment == 0 &&
inOutResourceDesc.Dimension == D3D12_RESOURCE_DIMENSION_BUFFER)
{
return {
outAllocInfo = {
AlignUp<UINT64>(inOutResourceDesc.Width, D3D12_DEFAULT_RESOURCE_PLACEMENT_ALIGNMENT), // SizeInBytes
D3D12_DEFAULT_RESOURCE_PLACEMENT_ALIGNMENT }; // Alignment
return S_OK;
}
#endif // #ifdef __ID3D12Device1_INTERFACE_DEFINED__
HRESULT hr = S_OK;
#if D3D12MA_USE_SMALL_RESOURCE_PLACEMENT_ALIGNMENT
if (inOutResourceDesc.Alignment == 0 &&
inOutResourceDesc.Dimension == D3D12_RESOURCE_DIMENSION_TEXTURE2D &&
@@ -7657,17 +7795,19 @@ D3D12_RESOURCE_ALLOCATION_INFO AllocatorPimpl::GetResourceAllocationInfo(D3D12_R
D3D12_SMALL_MSAA_RESOURCE_PLACEMENT_ALIGNMENT :
D3D12_SMALL_RESOURCE_PLACEMENT_ALIGNMENT;
inOutResourceDesc.Alignment = smallAlignmentToTry;
const D3D12_RESOURCE_ALLOCATION_INFO smallAllocInfo = GetResourceAllocationInfoNative(inOutResourceDesc);
hr = GetResourceAllocationInfoMiddle(
inOutResourceDesc, NumCastableFormats, pCastableFormats, outAllocInfo);
// Check if alignment requested has been granted.
if (smallAllocInfo.Alignment == smallAlignmentToTry)
if (SUCCEEDED(hr) && outAllocInfo.Alignment == smallAlignmentToTry)
{
return smallAllocInfo;
return S_OK;
}
inOutResourceDesc.Alignment = 0; // Restore original
}
#endif // #if D3D12MA_USE_SMALL_RESOURCE_PLACEMENT_ALIGNMENT
return GetResourceAllocationInfoNative(inOutResourceDesc);
return GetResourceAllocationInfoMiddle(
inOutResourceDesc, NumCastableFormats, pCastableFormats, outAllocInfo);
}
bool AllocatorPimpl::NewAllocationWithinBudget(D3D12_HEAP_TYPE heapType, UINT64 size)
@@ -7988,6 +8128,7 @@ HRESULT BlockVector::Allocate(
UINT64 size,
UINT64 alignment,
const ALLOCATION_DESC& allocDesc,
bool committedAllowed,
size_t allocationCount,
Allocation** pAllocations)
{
@@ -8002,6 +8143,7 @@ HRESULT BlockVector::Allocate(
size,
alignment,
allocDesc,
committedAllowed,
pAllocations + allocIndex);
if (FAILED(hr))
{
@@ -8090,40 +8232,43 @@ HRESULT BlockVector::CreateResource(
UINT64 alignment,
const ALLOCATION_DESC& allocDesc,
const CREATE_RESOURCE_PARAMS& createParams,
bool committedAllowed,
Allocation** ppAllocation,
REFIID riidResource,
void** ppvResource)
{
HRESULT hr = Allocate(size, alignment, allocDesc, 1, ppAllocation);
HRESULT hr = Allocate(size, alignment, allocDesc, committedAllowed, 1, ppAllocation);
if (FAILED(hr))
{
return hr;
}
ID3D12Resource* res = NULL;
hr = m_hAllocator->CreatePlacedResourceWrap(
(*ppAllocation)->m_Placed.block->GetHeap(),
(*ppAllocation)->GetOffset(),
createParams,
D3D12MA_IID_PPV_ARGS(&res));
if (SUCCEEDED(hr))
{
ID3D12Resource* res = NULL;
hr = m_hAllocator->CreatePlacedResourceWrap(
(*ppAllocation)->m_Placed.block->GetHeap(),
(*ppAllocation)->GetOffset(),
createParams,
D3D12MA_IID_PPV_ARGS(&res));
if (ppvResource != NULL)
{
hr = res->QueryInterface(riidResource, ppvResource);
}
if (SUCCEEDED(hr))
{
if (ppvResource != NULL)
{
hr = res->QueryInterface(riidResource, ppvResource);
}
if (SUCCEEDED(hr))
{
(*ppAllocation)->SetResourcePointer(res, createParams.GetBaseResourceDesc());
}
else
{
res->Release();
SAFE_RELEASE(*ppAllocation);
}
(*ppAllocation)->SetResourcePointer(res, createParams.GetBaseResourceDesc());
}
else
{
res->Release();
SAFE_RELEASE(*ppAllocation);
}
}
else
{
SAFE_RELEASE(*ppAllocation);
}
return hr;
}
@@ -8241,6 +8386,7 @@ HRESULT BlockVector::AllocatePage(
UINT64 size,
UINT64 alignment,
const ALLOCATION_DESC& allocDesc,
bool committedAllowed,
Allocation** pAllocation)
{
// Early reject: requested allocation size is larger that maximum block size for this block vector.
@@ -8257,13 +8403,19 @@ HRESULT BlockVector::AllocatePage(
freeMemory = (budget.UsageBytes < budget.BudgetBytes) ? (budget.BudgetBytes - budget.UsageBytes) : 0;
}
const bool canCreateNewBlock =
const bool canExceedFreeMemory = !committedAllowed;
bool canCreateNewBlock =
((allocDesc.Flags & ALLOCATION_FLAG_NEVER_ALLOCATE) == 0) &&
(m_Blocks.size() < m_MaxBlockCount) &&
// Even if we don't have to stay within budget with this allocation, when the
// budget would be exceeded, we don't want to allocate new blocks, but always
// create resources as committed.
freeMemory >= size;
(m_Blocks.size() < m_MaxBlockCount);
// Even if we don't have to stay within budget with this allocation, when the
// budget would be exceeded, we don't want to allocate new blocks, but always
// create resources as committed.
if (freeMemory < size && !canExceedFreeMemory)
{
canCreateNewBlock = false;
}
// 1. Search existing allocations
{
@@ -8313,26 +8465,29 @@ HRESULT BlockVector::AllocatePage(
}
}
size_t newBlockIndex = 0;
HRESULT hr = newBlockSize <= freeMemory ?
CreateBlock(newBlockSize, &newBlockIndex) : E_OUTOFMEMORY;
size_t newBlockIndex = SIZE_MAX;
HRESULT hr = E_OUTOFMEMORY;
if (newBlockSize <= freeMemory || canExceedFreeMemory)
{
hr = CreateBlock(newBlockSize, &newBlockIndex);
}
// Allocation of this size failed? Try 1/2, 1/4, 1/8 of m_PreferredBlockSize.
if (!m_ExplicitBlockSize)
{
while (FAILED(hr) && newBlockSizeShift < NEW_BLOCK_SIZE_SHIFT_MAX)
{
const UINT64 smallerNewBlockSize = newBlockSize / 2;
if (smallerNewBlockSize >= size)
{
newBlockSize = smallerNewBlockSize;
++newBlockSizeShift;
hr = newBlockSize <= freeMemory ?
CreateBlock(newBlockSize, &newBlockIndex) : E_OUTOFMEMORY;
}
else
if (smallerNewBlockSize < size)
{
break;
}
newBlockSize = smallerNewBlockSize;
++newBlockSizeShift;
if (newBlockSize <= freeMemory || canExceedFreeMemory)
{
hr = CreateBlock(newBlockSize, &newBlockIndex);
}
}
}
@@ -9509,6 +9664,8 @@ HRESULT Pool::BeginDefragmentation(const DEFRAGMENTATION_DESC* pDesc, Defragment
// Check for support
if (m_Pimpl->GetBlockVector()->GetAlgorithm() & POOL_FLAG_ALGORITHM_LINEAR)
return E_NOINTERFACE;
if(m_Pimpl->AlwaysCommitted())
return E_NOINTERFACE;
AllocatorPimpl* allocator = m_Pimpl->GetAllocator();
*ppContext = D3D12MA_NEW(allocator->GetAllocs(), DefragmentationContext)(allocator, *pDesc, m_Pimpl->GetBlockVector());
@@ -9612,7 +9769,7 @@ HRESULT Allocator::CreateResource3(
D3D12_BARRIER_LAYOUT InitialLayout,
const D3D12_CLEAR_VALUE* pOptimizedClearValue,
UINT32 NumCastableFormats,
DXGI_FORMAT* pCastableFormats,
const DXGI_FORMAT* pCastableFormats,
Allocation** ppAllocation,
REFIID riidResource,
void** ppvResource)
@@ -9702,7 +9859,7 @@ HRESULT Allocator::CreateAliasingResource2(
D3D12_BARRIER_LAYOUT InitialLayout,
const D3D12_CLEAR_VALUE* pOptimizedClearValue,
UINT32 NumCastableFormats,
DXGI_FORMAT* pCastableFormats,
const DXGI_FORMAT* pCastableFormats,
REFIID riidResource,
void** ppvResource)
{
@@ -9732,6 +9889,12 @@ HRESULT Allocator::CreatePool(
D3D12MA_ASSERT(0 && "Invalid arguments passed to Allocator::CreatePool.");
return E_INVALIDARG;
}
if ((pPoolDesc->Flags & POOL_FLAG_ALWAYS_COMMITTED) != 0 &&
(pPoolDesc->BlockSize != 0 || pPoolDesc->MinBlockCount > 0))
{
D3D12MA_ASSERT(0 && "Invalid arguments passed to Allocator::CreatePool while POOL_FLAG_ALWAYS_COMMITTED is specified.");
return E_INVALIDARG;
}
if (!m_Pimpl->HeapFlagsFulfillResourceHeapTier(pPoolDesc->HeapFlags))
{
D3D12MA_ASSERT(0 && "Invalid pPoolDesc->HeapFlags passed to Allocator::CreatePool. Did you forget to handle ResourceHeapTier=1?");

View File

@@ -8,3 +8,4 @@ Lénárd Szolnoki
Jan Pharago
Maya Warrier
Taha Khokhar
Anders Dalvander

View File

@@ -1,160 +1,233 @@
## fast_float number parsing library: 4x faster than strtod
[![Fuzzing Status](https://oss-fuzz-build-logs.storage.googleapis.com/badges/fast_float.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened&can=1&q=proj:fast_float)
[![Ubuntu 22.04 CI (GCC 11)](https://github.com/fastfloat/fast_float/actions/workflows/ubuntu22.yml/badge.svg)](https://github.com/fastfloat/fast_float/actions/workflows/ubuntu22.yml)
The fast_float library provides fast header-only implementations for the C++ from_chars
functions for `float` and `double` types as well as integer types. These functions convert ASCII strings representing decimal values (e.g., `1.3e10`) into binary types. We provide exact rounding (including
round to even). In our experience, these `fast_float` functions many times faster than comparable number-parsing functions from existing C++ standard libraries.
The fast_float library provides fast header-only implementations for the C++
from_chars functions for `float` and `double` types as well as integer types.
These functions convert ASCII strings representing decimal values (e.g.,
`1.3e10`) into binary types. We provide exact rounding (including round to
even). In our experience, these `fast_float` functions many times faster than
comparable number-parsing functions from existing C++ standard libraries.
Specifically, `fast_float` provides the following two functions to parse floating-point numbers with a C++17-like syntax (the library itself only requires C++11):
Specifically, `fast_float` provides the following two functions to parse
floating-point numbers with a C++17-like syntax (the library itself only
requires C++11):
```C++
from_chars_result from_chars(const char* first, const char* last, float& value, ...);
from_chars_result from_chars(const char* first, const char* last, double& value, ...);
from_chars_result from_chars(char const *first, char const *last, float &value, ...);
from_chars_result from_chars(char const *first, char const *last, double &value, ...);
```
You can also parse integer types:
```C++
from_chars_result from_chars(char const *first, char const *last, int &value, ...);
from_chars_result from_chars(char const *first, char const *last, unsigned &value, ...);
```
The return type (`from_chars_result`) is defined as the struct:
```C++
struct from_chars_result {
const char* ptr;
std::errc ec;
char const *ptr;
std::errc ec;
};
```
It parses the character sequence [first,last) for a number. It parses floating-point numbers expecting
a locale-independent format equivalent to the C++17 from_chars function.
The resulting floating-point value is the closest floating-point values (using either float or double),
using the "round to even" convention for values that would otherwise fall right in-between two values.
That is, we provide exact parsing according to the IEEE standard.
It parses the character sequence `[first, last)` for a number. It parses
floating-point numbers expecting a locale-independent format equivalent to the
C++17 from_chars function. The resulting floating-point value is the closest
floating-point values (using either `float` or `double`), using the "round to
even" convention for values that would otherwise fall right in-between two
values. That is, we provide exact parsing according to the IEEE standard.
Given a successful parse, the pointer (`ptr`) in the returned value is set to
point right after the parsed number, and the `value` referenced is set to the
parsed value. In case of error, the returned `ec` contains a representative
error, otherwise the default (`std::errc()`) value is stored.
Given a successful parse, the pointer (`ptr`) in the returned value is set to point right after the
parsed number, and the `value` referenced is set to the parsed value. In case of error, the returned
`ec` contains a representative error, otherwise the default (`std::errc()`) value is stored.
The implementation does not throw and does not allocate memory (e.g., with `new` or `malloc`).
The implementation does not throw and does not allocate memory (e.g., with `new`
or `malloc`).
It will parse infinity and nan values.
Example:
``` C++
```C++
#include "fast_float/fast_float.h"
#include <iostream>
int main() {
const std::string input = "3.1416 xyz ";
double result;
auto answer = fast_float::from_chars(input.data(), input.data()+input.size(), result);
if(answer.ec != std::errc()) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; }
std::cout << "parsed the number " << result << std::endl;
return EXIT_SUCCESS;
std::string input = "3.1416 xyz ";
double result;
auto answer = fast_float::from_chars(input.data(), input.data() + input.size(), result);
if (answer.ec != std::errc()) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; }
std::cout << "parsed the number " << result << std::endl;
return EXIT_SUCCESS;
}
```
You can parse delimited numbers:
```C++
const std::string input = "234532.3426362,7869234.9823,324562.645";
std::string input = "234532.3426362,7869234.9823,324562.645";
double result;
auto answer = fast_float::from_chars(input.data(), input.data()+input.size(), result);
if(answer.ec != std::errc()) {
auto answer = fast_float::from_chars(input.data(), input.data() + input.size(), result);
if (answer.ec != std::errc()) {
// check error
}
// we have result == 234532.3426362.
if(answer.ptr[0] != ',') {
if (answer.ptr[0] != ',') {
// unexpected delimiter
}
answer = fast_float::from_chars(answer.ptr + 1, input.data()+input.size(), result);
if(answer.ec != std::errc()) {
answer = fast_float::from_chars(answer.ptr + 1, input.data() + input.size(), result);
if (answer.ec != std::errc()) {
// check error
}
// we have result == 7869234.9823.
if(answer.ptr[0] != ',') {
if (answer.ptr[0] != ',') {
// unexpected delimiter
}
answer = fast_float::from_chars(answer.ptr + 1, input.data()+input.size(), result);
if(answer.ec != std::errc()) {
answer = fast_float::from_chars(answer.ptr + 1, input.data() + input.size(), result);
if (answer.ec != std::errc()) {
// check error
}
// we have result == 324562.645.
```
Like the C++17 standard, the `fast_float::from_chars` functions take an optional
last argument of the type `fast_float::chars_format`. It is a bitset value: we
check whether `fmt & fast_float::chars_format::fixed` and `fmt &
fast_float::chars_format::scientific` are set to determine whether we allow the
fixed point and scientific notation respectively. The default is
`fast_float::chars_format::general` which allows both `fixed` and `scientific`.
The library seeks to follow the C++17 (see
[28.2.3.(6.1)](https://eel.is/c++draft/charconv.from.chars#6.1)) specification.
Like the C++17 standard, the `fast_float::from_chars` functions take an optional last argument of
the type `fast_float::chars_format`. It is a bitset value: we check whether
`fmt & fast_float::chars_format::fixed` and `fmt & fast_float::chars_format::scientific` are set
to determine whether we allow the fixed point and scientific notation respectively.
The default is `fast_float::chars_format::general` which allows both `fixed` and `scientific`.
The library seeks to follow the C++17 (see [20.19.3](http://eel.is/c++draft/charconv.from.chars).(7.1)) specification.
* The `from_chars` function does not skip leading white-space characters.
* [A leading `+` sign](https://en.cppreference.com/w/cpp/utility/from_chars) is forbidden.
* It is generally impossible to represent a decimal value exactly as binary floating-point number (`float` and `double` types). We seek the nearest value. We round to an even mantissa when we are in-between two binary floating-point numbers.
* The `from_chars` function does not skip leading white-space characters (unless
`fast_float::chars_format::skip_white_space` is set).
* [A leading `+` sign](https://en.cppreference.com/w/cpp/utility/from_chars) is
forbidden (unless `fast_float::chars_format::allow_leading_plus` is set).
* It is generally impossible to represent a decimal value exactly as binary
floating-point number (`float` and `double` types). We seek the nearest value.
We round to an even mantissa when we are in-between two binary floating-point
numbers.
Furthermore, we have the following restrictions:
* We only support `float` and `double` types at this time.
* We support `float` and `double`, but not `long double`. We also support
fixed-width floating-point types such as `std::float64_t`, `std::float32_t`,
`std::float16_t`, and `std::bfloat16_t`.
* We only support the decimal format: we do not support hexadecimal strings.
* For values that are either very large or very small (e.g., `1e9999`), we represent it using the infinity or negative infinity value and the returned `ec` is set to `std::errc::result_out_of_range`.
* For values that are either very large or very small (e.g., `1e9999`), we
represent it using the infinity or negative infinity value and the returned
`ec` is set to `std::errc::result_out_of_range`.
We support Visual Studio, macOS, Linux, freeBSD. We support big and little endian. We support 32-bit and 64-bit systems.
We assume that the rounding mode is set to nearest (`std::fegetround() == FE_TONEAREST`).
We support Visual Studio, macOS, Linux, freeBSD. We support big and little
endian. We support 32-bit and 64-bit systems.
We assume that the rounding mode is set to nearest (`std::fegetround() ==
FE_TONEAREST`).
## Integer types
You can also parse integer types using different bases (e.g., 2, 10, 16). The following code will
print the number 22250738585072012 three times:
You can also parse integer types using different bases (e.g., 2, 10, 16). The
following code will print the number 22250738585072012 three times:
```C++
#include "fast_float/fast_float.h"
#include <iostream>
int main() {
uint64_t i;
const char str[] = "22250738585072012";
auto answer = fast_float::from_chars(str, str + strlen(str), i);
std::string str = "22250738585072012";
auto answer = fast_float::from_chars(str.data(), str.data() + str.size(), i);
if (answer.ec != std::errc()) {
std::cerr << "parsing failure\n";
return EXIT_FAILURE;
}
std::cout << "parsed the number "<< i << std::endl;
std::cout << "parsed the number " << i << std::endl;
const char binstr[] = "1001111000011001110110111001001010110100111000110001100";
std::string binstr = "1001111000011001110110111001001010110100111000110001100";
answer = fast_float::from_chars(binstr, binstr + strlen(binstr), i, 2);
answer = fast_float::from_chars(binstr.data(), binstr.data() + binstr.size(), i, 2);
if (answer.ec != std::errc()) {
std::cerr << "parsing failure\n";
return EXIT_FAILURE;
}
std::cout << "parsed the number "<< i << std::endl;
std::cout << "parsed the number " << i << std::endl;
std::string hexstr = "4f0cedc95a718c";
const char hexstr[] = "4f0cedc95a718c";
answer = fast_float::from_chars(hexstr, hexstr + strlen(hexstr), i, 16);
answer = fast_float::from_chars(hexstr.data(), hexstr.data() + hexstr.size(), i, 16);
if (answer.ec != std::errc()) {
std::cerr << "parsing failure\n";
return EXIT_FAILURE;
}
std::cout << "parsed the number "<< i << std::endl;
std::cout << "parsed the number " << i << std::endl;
return EXIT_SUCCESS;
}
```
## Behavior of result_out_of_range
When parsing floating-point values, the numbers can sometimes be too small
(e.g., `1e-1000`) or too large (e.g., `1e1000`). The C language established the
precedent that these small values are out of range. In such cases, it is
customary to parse small values to zero and large values to infinity. That is
the behaviour of the C language (e.g., `stdtod`). That is the behaviour followed
by the fast_float library.
Specifically, we follow Jonathan Wakely's interpretation of the standard:
> In any case, the resulting value is one of at most two floating-point values
> closest to the value of the string matching the pattern.
It is also the approach taken by the [Microsoft C++
library](https://github.com/microsoft/STL/blob/62205ab155d093e71dd9588a78f02c5396c3c14b/tests/std/tests/P0067R5_charconv/test.cpp#L943-L946).
Hence, we have the following examples:
```cpp
double result = -1;
std::string str = "3e-1000";
auto r = fast_float::from_chars(str.data(), str.data() + str.size(), result);
// r.ec == std::errc::result_out_of_range
// r.ptr == str.data() + 7
// result == 0
```
```cpp
double result = -1;
std::string str = "3e1000";
auto r = fast_float::from_chars(str.data(), str.data() + str.size(), result);
// r.ec == std::errc::result_out_of_range
// r.ptr == str.data() + 6
// result == std::numeric_limits<double>::infinity()
```
Users who wish for the value to be left unmodified given
`std::errc::result_out_of_range` may do so by adding two lines of code:
```cpp
double old_result = result; // make copy
auto r = fast_float::from_chars(start, end, result);
if (r.ec == std::errc::result_out_of_range) { result = old_result; }
```
## C++20: compile-time evaluation (constexpr)
In C++20, you may use `fast_float::from_chars` to parse strings
at compile-time, as in the following example:
In C++20, you may use `fast_float::from_chars` to parse strings at compile-time,
as in the following example:
```C++
// consteval forces compile-time evaluation of the function in C++20.
consteval double parse(std::string_view input) {
double result;
auto answer = fast_float::from_chars(input.data(), input.data()+input.size(), result);
if(answer.ec != std::errc()) { return -1.0; }
auto answer = fast_float::from_chars(input.data(), input.data() + input.size(), result);
if (answer.ec != std::errc()) { return -1.0; }
return result;
}
@@ -167,108 +240,107 @@ constexpr double constexptest() {
## C++23: Fixed width floating-point types
The library also supports fixed-width floating-point types such as `std::float32_t` and `std::float64_t`. E.g., you can write:
The library also supports fixed-width floating-point types such as
`std::float64_t`, `std::float32_t`, `std::float16_t`, and `std::bfloat16_t`.
E.g., you can write:
```C++
std::float32_t result;
auto answer = fast_float::from_chars(f.data(), f.data() + f.size(), result);
``````
```
## Non-ASCII Inputs
We also support UTF-16 and UTF-32 inputs, as well as ASCII/UTF-8, as in the following example:
We also support UTF-16 and UTF-32 inputs, as well as ASCII/UTF-8, as in the
following example:
``` C++
```C++
#include "fast_float/fast_float.h"
#include <iostream>
int main() {
const std::u16string input = u"3.1416 xyz ";
double result;
auto answer = fast_float::from_chars(input.data(), input.data()+input.size(), result);
if(answer.ec != std::errc()) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; }
std::cout << "parsed the number " << result << std::endl;
return EXIT_SUCCESS;
std::u16string input = u"3.1416 xyz ";
double result;
auto answer = fast_float::from_chars(input.data(), input.data() + input.size(), result);
if (answer.ec != std::errc()) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; }
std::cout << "parsed the number " << result << std::endl;
return EXIT_SUCCESS;
}
```
## Advanced options: using commas as decimal separator, JSON and Fortran
## Advanced options: using commas as decimal separator, JSON and Fortran
The C++ standard stipulate that `from_chars` has to be locale-independent. In
particular, the decimal separator has to be the period (`.`). However,
some users still want to use the `fast_float` library with in a locale-dependent
manner. Using a separate function called `from_chars_advanced`, we allow the users
to pass a `parse_options` instance which contains a custom decimal separator (e.g.,
the comma). You may use it as follows.
particular, the decimal separator has to be the period (`.`). However, some
users still want to use the `fast_float` library with in a locale-dependent
manner. Using a separate function called `from_chars_advanced`, we allow the
users to pass a `parse_options` instance which contains a custom decimal
separator (e.g., the comma). You may use it as follows.
```C++
#include "fast_float/fast_float.h"
#include <iostream>
int main() {
const std::string input = "3,1416 xyz ";
double result;
fast_float::parse_options options{fast_float::chars_format::general, ','};
auto answer = fast_float::from_chars_advanced(input.data(), input.data()+input.size(), result, options);
if((answer.ec != std::errc()) || ((result != 3.1416))) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; }
std::cout << "parsed the number " << result << std::endl;
return EXIT_SUCCESS;
std::string input = "3,1416 xyz ";
double result;
fast_float::parse_options options{fast_float::chars_format::general, ','};
auto answer = fast_float::from_chars_advanced(input.data(), input.data() + input.size(), result, options);
if ((answer.ec != std::errc()) || ((result != 3.1416))) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; }
std::cout << "parsed the number " << result << std::endl;
return EXIT_SUCCESS;
}
```
You can also parse Fortran-like inputs:
### You can also parse Fortran-like inputs
```C++
#include "fast_float/fast_float.h"
#include <iostream>
int main() {
const std::string input = "1d+4";
double result;
fast_float::parse_options options{ fast_float::chars_format::fortran };
auto answer = fast_float::from_chars_advanced(input.data(), input.data()+input.size(), result, options);
if((answer.ec != std::errc()) || ((result != 10000))) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; }
std::cout << "parsed the number " << result << std::endl;
return EXIT_SUCCESS;
std::string input = "1d+4";
double result;
fast_float::parse_options options{fast_float::chars_format::fortran};
auto answer = fast_float::from_chars_advanced(input.data(), input.data() + input.size(), result, options);
if ((answer.ec != std::errc()) || ((result != 10000))) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; }
std::cout << "parsed the number " << result << std::endl;
return EXIT_SUCCESS;
}
```
You may also enforce the JSON format ([RFC 8259](https://datatracker.ietf.org/doc/html/rfc8259#section-6)):
### You may also enforce the JSON format ([RFC 8259](https://datatracker.ietf.org/doc/html/rfc8259#section-6))
```C++
#include "fast_float/fast_float.h"
#include <iostream>
int main() {
const std::string input = "+.1"; // not valid
double result;
fast_float::parse_options options{ fast_float::chars_format::json };
auto answer = fast_float::from_chars_advanced(input.data(), input.data()+input.size(), result, options);
if(answer.ec == std::errc()) { std::cerr << "should have failed\n"; return EXIT_FAILURE; }
return EXIT_SUCCESS;
std::string input = "+.1"; // not valid
double result;
fast_float::parse_options options{fast_float::chars_format::json};
auto answer = fast_float::from_chars_advanced(input.data(), input.data() + input.size(), result, options);
if (answer.ec == std::errc()) { std::cerr << "should have failed\n"; return EXIT_FAILURE; }
return EXIT_SUCCESS;
}
```
By default the JSON format does not allow `inf`:
```C++
#include "fast_float/fast_float.h"
#include <iostream>
int main() {
const std::string input = "inf"; // not valid in JSON
double result;
fast_float::parse_options options{ fast_float::chars_format::json };
auto answer = fast_float::from_chars_advanced(input.data(), input.data()+input.size(), result, options);
if(answer.ec == std::errc()) { std::cerr << "should have failed\n"; return EXIT_FAILURE; }
std::string input = "inf"; // not valid in JSON
double result;
fast_float::parse_options options{fast_float::chars_format::json};
auto answer = fast_float::from_chars_advanced(input.data(), input.data() + input.size(), result, options);
if (answer.ec == std::errc()) { std::cerr << "should have failed\n"; return EXIT_FAILURE; }
return EXIT_SUCCESS;
}
```
You can allow it with a non-standard `json_or_infnan` variant:
```C++
@@ -276,55 +348,77 @@ You can allow it with a non-standard `json_or_infnan` variant:
#include <iostream>
int main() {
const std::string input = "inf"; // not valid in JSON but we allow it with json_or_infnan
double result;
fast_float::parse_options options{ fast_float::chars_format::json_or_infnan };
auto answer = fast_float::from_chars_advanced(input.data(), input.data()+input.size(), result, options);
if(answer.ec != std::errc() || (!std::isinf(result))) { std::cerr << "should have parsed infinity\n"; return EXIT_FAILURE; }
return EXIT_SUCCESS;
std::string input = "inf"; // not valid in JSON but we allow it with json_or_infnan
double result;
fast_float::parse_options options{fast_float::chars_format::json_or_infnan};
auto answer = fast_float::from_chars_advanced(input.data(), input.data() + input.size(), result, options);
if (answer.ec != std::errc() || (!std::isinf(result))) { std::cerr << "should have parsed infinity\n"; return EXIT_FAILURE; }
return EXIT_SUCCESS;
}
``````
```
## Relation With Other Work
## Users and Related Work
The fast_float library is part of:
- GCC (as of version 12): the `from_chars` function in GCC relies on fast_float.
- [WebKit](https://github.com/WebKit/WebKit), the engine behind Safari (Apple's web browser)
* GCC (as of version 12): the `from_chars` function in GCC relies on fast_float,
* [Chromium](https://github.com/Chromium/Chromium), the engine behind Google
Chrome, Microsoft Edge, and Opera,
* [WebKit](https://github.com/WebKit/WebKit), the engine behind Safari (Apple's
web browser),
* [DuckDB](https://duckdb.org),
* [Redis](https://github.com/redis/redis) and [Valkey](https://github.com/valkey-io/valkey),
* [Apache Arrow](https://github.com/apache/arrow/pull/8494) where it multiplied
the number parsing speed by two or three times,
* [Google Jsonnet](https://github.com/google/jsonnet),
* [ClickHouse](https://github.com/ClickHouse/ClickHouse).
The fastfloat algorithm is part of the [LLVM standard
libraries](https://github.com/llvm/llvm-project/commit/87c016078ad72c46505461e4ff8bfa04819fe7ba).
There is a [derived implementation part of
AdaCore](https://github.com/AdaCore/VSS).
The fastfloat algorithm is part of the [LLVM standard libraries](https://github.com/llvm/llvm-project/commit/87c016078ad72c46505461e4ff8bfa04819fe7ba).
There is a [derived implementation part of AdaCore](https://github.com/AdaCore/VSS).
The fast_float library provides a performance similar to that of the [fast_double_parser](https://github.com/lemire/fast_double_parser) library but using an updated algorithm reworked from the ground up, and while offering an API more in line with the expectations of C++ programmers. The fast_double_parser library is part of the [Microsoft LightGBM machine-learning framework](https://github.com/microsoft/LightGBM).
The fast_float library provides a performance similar to that of the
[fast_double_parser](https://github.com/lemire/fast_double_parser) library but
using an updated algorithm reworked from the ground up, and while offering an
API more in line with the expectations of C++ programmers. The
fast_double_parser library is part of the [Microsoft LightGBM machine-learning
framework](https://github.com/microsoft/LightGBM).
## References
- Daniel Lemire, [Number Parsing at a Gigabyte per Second](https://arxiv.org/abs/2101.11408), Software: Practice and Experience 51 (8), 2021.
- Noble Mushtak, Daniel Lemire, [Fast Number Parsing Without Fallback](https://arxiv.org/abs/2212.06644), Software: Practice and Experience 53 (7), 2023.
* Daniel Lemire, [Number Parsing at a Gigabyte per
Second](https://arxiv.org/abs/2101.11408), Software: Practice and Experience
51 (8), 2021.
* Noble Mushtak, Daniel Lemire, [Fast Number Parsing Without
Fallback](https://arxiv.org/abs/2212.06644), Software: Practice and Experience
53 (7), 2023.
## Other programming languages
- [There is an R binding](https://github.com/eddelbuettel/rcppfastfloat) called `rcppfastfloat`.
- [There is a Rust port of the fast_float library](https://github.com/aldanor/fast-float-rust/) called `fast-float-rust`.
- [There is a Java port of the fast_float library](https://github.com/wrandelshofer/FastDoubleParser) called `FastDoubleParser`. It used for important systems such as [Jackson](https://github.com/FasterXML/jackson-core).
- [There is a C# port of the fast_float library](https://github.com/CarlVerret/csFastFloat) called `csFastFloat`.
## Users
The fast_float library is used by [Apache Arrow](https://github.com/apache/arrow/pull/8494) where it multiplied the number parsing speed by two or three times. It is also used by [ClickHouse](https://github.com/ClickHouse/ClickHouse) and by [Google Jsonnet](https://github.com/google/jsonnet). It is part of GCC (as of GCC 12). It is part of WebKit (Safari).
* [There is an R binding](https://github.com/eddelbuettel/rcppfastfloat) called
`rcppfastfloat`.
* [There is a Rust port of the fast_float
library](https://github.com/aldanor/fast-float-rust/) called
`fast-float-rust`.
* [There is a Java port of the fast_float
library](https://github.com/wrandelshofer/FastDoubleParser) called
`FastDoubleParser`. It used for important systems such as
[Jackson](https://github.com/FasterXML/jackson-core).
* [There is a C# port of the fast_float
library](https://github.com/CarlVerret/csFastFloat) called `csFastFloat`.
## How fast is it?
It can parse random floating-point numbers at a speed of 1 GB/s on some systems. We find that it is often twice as fast as the best available competitor, and many times faster than many standard-library implementations.
It can parse random floating-point numbers at a speed of 1 GB/s on some systems.
We find that it is often twice as fast as the best available competitor, and
many times faster than many standard-library implementations.
<img src="http://lemire.me/blog/wp-content/uploads/2020/11/fastfloat_speed.png" width="400">
<img src="https://lemire.me/blog/wp-content/uploads/2020/11/fastfloat_speed.png"
width="400" alt="fast_float is many times faster than many standard-library
implementations">
```
```bash
$ ./build/benchmarks/benchmark
# parsing random integers in the range [0,1)
volume = 2.09808 MB
@@ -335,75 +429,122 @@ abseil : 430.45 MB/s (+/- 2.2 %) 20.52 Mfl
fastfloat : 1042.38 MB/s (+/- 9.9 %) 49.68 Mfloat/s
```
See https://github.com/lemire/simple_fastfloat_benchmark for our benchmarking code.
See the [Benchmarking](#benchmarking) section for instructions on how to run our benchmarks.
## Video
[![Go Systems 2020](http://img.youtube.com/vi/AVXgvlMeIm4/0.jpg)](http://www.youtube.com/watch?v=AVXgvlMeIm4)<br />
[![Go Systems 2020](https://img.youtube.com/vi/AVXgvlMeIm4/0.jpg)](https://www.youtube.com/watch?v=AVXgvlMeIm4)
## Using as a CMake dependency
This library is header-only by design. The CMake file provides the `fast_float` target
which is merely a pointer to the `include` directory.
This library is header-only by design. The CMake file provides the `fast_float`
target which is merely a pointer to the `include` directory.
If you drop the `fast_float` repository in your CMake project, you should be able to use
it in this manner:
If you drop the `fast_float` repository in your CMake project, you should be
able to use it in this manner:
```cmake
add_subdirectory(fast_float)
target_link_libraries(myprogram PUBLIC fast_float)
```
Or you may want to retrieve the dependency automatically if you have a sufficiently recent version of CMake (3.11 or better at least):
Or you may want to retrieve the dependency automatically if you have a
sufficiently recent version of CMake (3.11 or better at least):
```cmake
FetchContent_Declare(
fast_float
GIT_REPOSITORY https://github.com/lemire/fast_float.git
GIT_TAG tags/v1.1.2
GIT_REPOSITORY https://github.com/fastfloat/fast_float.git
GIT_TAG tags/v8.0.2
GIT_SHALLOW TRUE)
FetchContent_MakeAvailable(fast_float)
target_link_libraries(myprogram PUBLIC fast_float)
```
You should change the `GIT_TAG` line so that you recover the version you wish to use.
You should change the `GIT_TAG` line so that you recover the version you wish to
use.
You may also use [CPM](https://github.com/cpm-cmake/CPM.cmake), like so:
```cmake
CPMAddPackage(
NAME fast_float
GITHUB_REPOSITORY "fastfloat/fast_float"
GIT_TAG v8.0.2)
```
## Using as single header
The script `script/amalgamate.py` may be used to generate a single header
version of the library if so desired.
Just run the script from the root directory of this repository.
You can customize the license type and output file if desired as described in
the command line help.
version of the library if so desired. Just run the script from the root
directory of this repository. You can customize the license type and output file
if desired as described in the command line help.
You may directly download automatically generated single-header files:
https://github.com/fastfloat/fast_float/releases/download/v6.1.1/fast_float.h
<https://github.com/fastfloat/fast_float/releases/download/v8.0.2/fast_float.h>
## RFC 7159
## Benchmarking
If you need support for RFC 7159 (JSON standard), you may want to consider using the [fast_double_parser](https://github.com/lemire/fast_double_parser/) library instead.
The project has its own benchmarks with realistic data inputs. Under Linux or macOS,
you can use it as follows if your system supports C++17:
```
cmake -B build -D FASTFLOAT_BENCHMARKS=ON
cmake --build build
./build/benchmarks/realbenchmark
```
Importantly, by default, the benchmark is built in Release mode.
The instructions are similar under Windows.
Under Linux and macOS, it is recommended to run the benchmarks in a privileged manner to get access
to hardware performance counters. You may be able to do so with the `sudo` command
in some cases:
```
sudo ./build/benchmarks/realbenchmark
```
If you have a text file containing one number per line (`myfile.txt`), you can run a benchmark over it like so:
```
cmake -B build -D FASTFLOAT_BENCHMARKS=ON
cmake --build build
./build/benchmarks/realbenchmark myfile.txt
```
## Packages
* The fast_float library is part of the [Conan package
manager](https://conan.io/center/recipes/fast_float).
* It is part of the [brew package
manager](https://formulae.brew.sh/formula/fast_float).
* Some Linux distribution like Fedora include fast_float (e.g., as
`fast_float-devel`).
## Credit
Though this work is inspired by many different people, this work benefited especially from exchanges with
Michael Eisel, who motivated the original research with his key insights, and with Nigel Tao who provided
invaluable feedback. Rémy Oudompheng first implemented a fast path we use in the case of long digits.
Though this work is inspired by many different people, this work benefited
especially from exchanges with Michael Eisel, who motivated the original
research with his key insights, and with Nigel Tao who provided invaluable
feedback. Rémy Oudompheng first implemented a fast path we use in the case of
long digits.
The library includes code adapted from Google Wuffs (written by Nigel Tao) which was originally published
under the Apache 2.0 license.
The library includes code adapted from Google Wuffs (written by Nigel Tao) which
was originally published under the Apache 2.0 license.
## License
<sup>
Licensed under either of <a href="LICENSE-APACHE">Apache License, Version
2.0</a> or <a href="LICENSE-MIT">MIT license</a> or <a href="LICENSE-BOOST">BOOST license</a> .
2.0</a> or <a href="LICENSE-MIT">MIT license</a> or <a
href="LICENSE-BOOST">BOOST license</a>.
</sup>
<br>
<br/>
<sub>
Unless you explicitly state otherwise, any contribution intentionally submitted

View File

@@ -20,8 +20,7 @@
namespace fast_float {
template <typename UC>
fastfloat_really_inline constexpr bool has_simd_opt() {
template <typename UC> fastfloat_really_inline constexpr bool has_simd_opt() {
#ifdef FASTFLOAT_HAS_SIMD
return std::is_same<UC, char16_t>::value;
#else
@@ -37,24 +36,20 @@ fastfloat_really_inline constexpr bool is_integer(UC c) noexcept {
}
fastfloat_really_inline constexpr uint64_t byteswap(uint64_t val) {
return (val & 0xFF00000000000000) >> 56
| (val & 0x00FF000000000000) >> 40
| (val & 0x0000FF0000000000) >> 24
| (val & 0x000000FF00000000) >> 8
| (val & 0x00000000FF000000) << 8
| (val & 0x0000000000FF0000) << 24
| (val & 0x000000000000FF00) << 40
| (val & 0x00000000000000FF) << 56;
return (val & 0xFF00000000000000) >> 56 | (val & 0x00FF000000000000) >> 40 |
(val & 0x0000FF0000000000) >> 24 | (val & 0x000000FF00000000) >> 8 |
(val & 0x00000000FF000000) << 8 | (val & 0x0000000000FF0000) << 24 |
(val & 0x000000000000FF00) << 40 | (val & 0x00000000000000FF) << 56;
}
// Read 8 UC into a u64. Truncates UC if not char.
template <typename UC>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
uint64_t read8_to_u64(const UC *chars) {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 uint64_t
read8_to_u64(UC const *chars) {
if (cpp20_and_in_constexpr() || !std::is_same<UC, char>::value) {
uint64_t val = 0;
for(int i = 0; i < 8; ++i) {
val |= uint64_t(uint8_t(*chars)) << (i*8);
for (int i = 0; i < 8; ++i) {
val |= uint64_t(uint8_t(*chars)) << (i * 8);
++chars;
}
return val;
@@ -70,44 +65,41 @@ uint64_t read8_to_u64(const UC *chars) {
#ifdef FASTFLOAT_SSE2
fastfloat_really_inline
uint64_t simd_read8_to_u64(const __m128i data) {
FASTFLOAT_SIMD_DISABLE_WARNINGS
const __m128i packed = _mm_packus_epi16(data, data);
fastfloat_really_inline uint64_t simd_read8_to_u64(__m128i const data) {
FASTFLOAT_SIMD_DISABLE_WARNINGS
__m128i const packed = _mm_packus_epi16(data, data);
#ifdef FASTFLOAT_64BIT
return uint64_t(_mm_cvtsi128_si64(packed));
#else
uint64_t value;
// Visual Studio + older versions of GCC don't support _mm_storeu_si64
_mm_storel_epi64(reinterpret_cast<__m128i*>(&value), packed);
_mm_storel_epi64(reinterpret_cast<__m128i *>(&value), packed);
return value;
#endif
FASTFLOAT_SIMD_RESTORE_WARNINGS
FASTFLOAT_SIMD_RESTORE_WARNINGS
}
fastfloat_really_inline
uint64_t simd_read8_to_u64(const char16_t* chars) {
FASTFLOAT_SIMD_DISABLE_WARNINGS
return simd_read8_to_u64(_mm_loadu_si128(reinterpret_cast<const __m128i*>(chars)));
FASTFLOAT_SIMD_RESTORE_WARNINGS
fastfloat_really_inline uint64_t simd_read8_to_u64(char16_t const *chars) {
FASTFLOAT_SIMD_DISABLE_WARNINGS
return simd_read8_to_u64(
_mm_loadu_si128(reinterpret_cast<__m128i const *>(chars)));
FASTFLOAT_SIMD_RESTORE_WARNINGS
}
#elif defined(FASTFLOAT_NEON)
fastfloat_really_inline
uint64_t simd_read8_to_u64(const uint16x8_t data) {
FASTFLOAT_SIMD_DISABLE_WARNINGS
fastfloat_really_inline uint64_t simd_read8_to_u64(uint16x8_t const data) {
FASTFLOAT_SIMD_DISABLE_WARNINGS
uint8x8_t utf8_packed = vmovn_u16(data);
return vget_lane_u64(vreinterpret_u64_u8(utf8_packed), 0);
FASTFLOAT_SIMD_RESTORE_WARNINGS
FASTFLOAT_SIMD_RESTORE_WARNINGS
}
fastfloat_really_inline
uint64_t simd_read8_to_u64(const char16_t* chars) {
FASTFLOAT_SIMD_DISABLE_WARNINGS
return simd_read8_to_u64(vld1q_u16(reinterpret_cast<const uint16_t*>(chars)));
FASTFLOAT_SIMD_RESTORE_WARNINGS
fastfloat_really_inline uint64_t simd_read8_to_u64(char16_t const *chars) {
FASTFLOAT_SIMD_DISABLE_WARNINGS
return simd_read8_to_u64(
vld1q_u16(reinterpret_cast<uint16_t const *>(chars)));
FASTFLOAT_SIMD_RESTORE_WARNINGS
}
#endif // FASTFLOAT_SSE2
@@ -119,101 +111,84 @@ template <typename UC>
template <typename UC, FASTFLOAT_ENABLE_IF(!has_simd_opt<UC>()) = 0>
#endif
// dummy for compile
uint64_t simd_read8_to_u64(UC const*) {
uint64_t simd_read8_to_u64(UC const *) {
return 0;
}
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
void write_u64(uint8_t *chars, uint64_t val) {
if (cpp20_and_in_constexpr()) {
for(int i = 0; i < 8; ++i) {
*chars = uint8_t(val);
val >>= 8;
++chars;
}
return;
}
#if FASTFLOAT_IS_BIG_ENDIAN == 1
// Need to read as-if the number was in little-endian order.
val = byteswap(val);
#endif
::memcpy(chars, &val, sizeof(uint64_t));
}
// credit @aqrit
fastfloat_really_inline FASTFLOAT_CONSTEXPR14
uint32_t parse_eight_digits_unrolled(uint64_t val) {
const uint64_t mask = 0x000000FF000000FF;
const uint64_t mul1 = 0x000F424000000064; // 100 + (1000000ULL << 32)
const uint64_t mul2 = 0x0000271000000001; // 1 + (10000ULL << 32)
fastfloat_really_inline FASTFLOAT_CONSTEXPR14 uint32_t
parse_eight_digits_unrolled(uint64_t val) {
uint64_t const mask = 0x000000FF000000FF;
uint64_t const mul1 = 0x000F424000000064; // 100 + (1000000ULL << 32)
uint64_t const mul2 = 0x0000271000000001; // 1 + (10000ULL << 32)
val -= 0x3030303030303030;
val = (val * 10) + (val >> 8); // val = (val * 2561) >> 8;
val = (((val & mask) * mul1) + (((val >> 16) & mask) * mul2)) >> 32;
return uint32_t(val);
}
// Call this if chars are definitely 8 digits.
template <typename UC>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
uint32_t parse_eight_digits_unrolled(UC const * chars) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 uint32_t
parse_eight_digits_unrolled(UC const *chars) noexcept {
if (cpp20_and_in_constexpr() || !has_simd_opt<UC>()) {
return parse_eight_digits_unrolled(read8_to_u64(chars)); // truncation okay
}
return parse_eight_digits_unrolled(simd_read8_to_u64(chars));
}
// credit @aqrit
fastfloat_really_inline constexpr bool is_made_of_eight_digits_fast(uint64_t val) noexcept {
fastfloat_really_inline constexpr bool
is_made_of_eight_digits_fast(uint64_t val) noexcept {
return !((((val + 0x4646464646464646) | (val - 0x3030303030303030)) &
0x8080808080808080));
0x8080808080808080));
}
#ifdef FASTFLOAT_HAS_SIMD
// Call this if chars might not be 8 digits.
// Using this style (instead of is_made_of_eight_digits_fast() then parse_eight_digits_unrolled())
// ensures we don't load SIMD registers twice.
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
bool simd_parse_if_eight_digits_unrolled(const char16_t* chars, uint64_t& i) noexcept {
// Using this style (instead of is_made_of_eight_digits_fast() then
// parse_eight_digits_unrolled()) ensures we don't load SIMD registers twice.
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 bool
simd_parse_if_eight_digits_unrolled(char16_t const *chars,
uint64_t &i) noexcept {
if (cpp20_and_in_constexpr()) {
return false;
}
}
#ifdef FASTFLOAT_SSE2
FASTFLOAT_SIMD_DISABLE_WARNINGS
const __m128i data = _mm_loadu_si128(reinterpret_cast<const __m128i*>(chars));
FASTFLOAT_SIMD_DISABLE_WARNINGS
__m128i const data =
_mm_loadu_si128(reinterpret_cast<__m128i const *>(chars));
// (x - '0') <= 9
// http://0x80.pl/articles/simd-parsing-int-sequences.html
const __m128i t0 = _mm_add_epi16(data, _mm_set1_epi16(32720));
const __m128i t1 = _mm_cmpgt_epi16(t0, _mm_set1_epi16(-32759));
__m128i const t0 = _mm_add_epi16(data, _mm_set1_epi16(32720));
__m128i const t1 = _mm_cmpgt_epi16(t0, _mm_set1_epi16(-32759));
if (_mm_movemask_epi8(t1) == 0) {
i = i * 100000000 + parse_eight_digits_unrolled(simd_read8_to_u64(data));
return true;
}
else return false;
FASTFLOAT_SIMD_RESTORE_WARNINGS
} else
return false;
FASTFLOAT_SIMD_RESTORE_WARNINGS
#elif defined(FASTFLOAT_NEON)
FASTFLOAT_SIMD_DISABLE_WARNINGS
const uint16x8_t data = vld1q_u16(reinterpret_cast<const uint16_t*>(chars));
FASTFLOAT_SIMD_DISABLE_WARNINGS
uint16x8_t const data = vld1q_u16(reinterpret_cast<uint16_t const *>(chars));
// (x - '0') <= 9
// http://0x80.pl/articles/simd-parsing-int-sequences.html
const uint16x8_t t0 = vsubq_u16(data, vmovq_n_u16('0'));
const uint16x8_t mask = vcltq_u16(t0, vmovq_n_u16('9' - '0' + 1));
uint16x8_t const t0 = vsubq_u16(data, vmovq_n_u16('0'));
uint16x8_t const mask = vcltq_u16(t0, vmovq_n_u16('9' - '0' + 1));
if (vminvq_u16(mask) == 0xFFFF) {
i = i * 100000000 + parse_eight_digits_unrolled(simd_read8_to_u64(data));
return true;
}
else return false;
FASTFLOAT_SIMD_RESTORE_WARNINGS
} else
return false;
FASTFLOAT_SIMD_RESTORE_WARNINGS
#else
(void)chars; (void)i;
(void)chars;
(void)i;
return false;
#endif // FASTFLOAT_SSE2
}
@@ -227,79 +202,119 @@ template <typename UC>
template <typename UC, FASTFLOAT_ENABLE_IF(!has_simd_opt<UC>()) = 0>
#endif
// dummy for compile
bool simd_parse_if_eight_digits_unrolled(UC const*, uint64_t&) {
bool simd_parse_if_eight_digits_unrolled(UC const *, uint64_t &) {
return 0;
}
template <typename UC, FASTFLOAT_ENABLE_IF(!std::is_same<UC, char>::value) = 0>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
void loop_parse_if_eight_digits(const UC*& p, const UC* const pend, uint64_t& i) {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 void
loop_parse_if_eight_digits(UC const *&p, UC const *const pend, uint64_t &i) {
if (!has_simd_opt<UC>()) {
return;
}
while ((std::distance(p, pend) >= 8) && simd_parse_if_eight_digits_unrolled(p, i)) { // in rare cases, this will overflow, but that's ok
while ((std::distance(p, pend) >= 8) &&
simd_parse_if_eight_digits_unrolled(
p, i)) { // in rare cases, this will overflow, but that's ok
p += 8;
}
}
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
void loop_parse_if_eight_digits(const char*& p, const char* const pend, uint64_t& i) {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 void
loop_parse_if_eight_digits(char const *&p, char const *const pend,
uint64_t &i) {
// optimizes better than parse_if_eight_digits_unrolled() for UC = char.
while ((std::distance(p, pend) >= 8) && is_made_of_eight_digits_fast(read8_to_u64(p))) {
i = i * 100000000 + parse_eight_digits_unrolled(read8_to_u64(p)); // in rare cases, this will overflow, but that's ok
while ((std::distance(p, pend) >= 8) &&
is_made_of_eight_digits_fast(read8_to_u64(p))) {
i = i * 100000000 +
parse_eight_digits_unrolled(read8_to_u64(
p)); // in rare cases, this will overflow, but that's ok
p += 8;
}
}
template <typename UC>
struct parsed_number_string_t {
enum class parse_error {
no_error,
// [JSON-only] The minus sign must be followed by an integer.
missing_integer_after_sign,
// A sign must be followed by an integer or dot.
missing_integer_or_dot_after_sign,
// [JSON-only] The integer part must not have leading zeros.
leading_zeros_in_integer_part,
// [JSON-only] The integer part must have at least one digit.
no_digits_in_integer_part,
// [JSON-only] If there is a decimal point, there must be digits in the
// fractional part.
no_digits_in_fractional_part,
// The mantissa must have at least one digit.
no_digits_in_mantissa,
// Scientific notation requires an exponential part.
missing_exponential_part,
};
template <typename UC> struct parsed_number_string_t {
int64_t exponent{0};
uint64_t mantissa{0};
UC const * lastmatch{nullptr};
UC const *lastmatch{nullptr};
bool negative{false};
bool valid{false};
bool too_many_digits{false};
// contains the range of the significant digits
span<const UC> integer{}; // non-nullable
span<const UC> fraction{}; // nullable
span<UC const> integer{}; // non-nullable
span<UC const> fraction{}; // nullable
parse_error error{parse_error::no_error};
};
using byte_span = span<const char>;
using byte_span = span<char const>;
using parsed_number_string = parsed_number_string_t<char>;
template <typename UC>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 parsed_number_string_t<UC>
report_parse_error(UC const *p, parse_error error) {
parsed_number_string_t<UC> answer;
answer.valid = false;
answer.lastmatch = p;
answer.error = error;
return answer;
}
// Assuming that you use no more than 19 digits, this will
// parse an ASCII string.
template <typename UC>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
parsed_number_string_t<UC> parse_number_string(UC const *p, UC const * pend, parse_options_t<UC> options) noexcept {
chars_format const fmt = options.format;
template <bool basic_json_fmt, typename UC>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 parsed_number_string_t<UC>
parse_number_string(UC const *p, UC const *pend,
parse_options_t<UC> options) noexcept {
chars_format const fmt = detail::adjust_for_feature_macros(options.format);
UC const decimal_point = options.decimal_point;
parsed_number_string_t<UC> answer;
answer.valid = false;
answer.too_many_digits = false;
// assume p < pend, so dereference without checks;
answer.negative = (*p == UC('-'));
#ifdef FASTFLOAT_ALLOWS_LEADING_PLUS // disabled by default
if ((*p == UC('-')) || (!(fmt & FASTFLOAT_JSONFMT) && *p == UC('+'))) {
#else
if (*p == UC('-')) { // C++17 20.19.3.(7.1) explicitly forbids '+' sign here
#endif
// C++17 20.19.3.(7.1) explicitly forbids '+' sign here
if ((*p == UC('-')) || (uint64_t(fmt & chars_format::allow_leading_plus) &&
!basic_json_fmt && *p == UC('+'))) {
++p;
if (p == pend) {
return answer;
return report_parse_error<UC>(
p, parse_error::missing_integer_or_dot_after_sign);
}
if (fmt & FASTFLOAT_JSONFMT) {
FASTFLOAT_IF_CONSTEXPR17(basic_json_fmt) {
if (!is_integer(*p)) { // a sign must be followed by an integer
return answer;
}
} else {
if (!is_integer(*p) && (*p != decimal_point)) { // a sign must be followed by an integer or the dot
return answer;
return report_parse_error<UC>(p,
parse_error::missing_integer_after_sign);
}
}
else {
if (!is_integer(*p) &&
(*p !=
decimal_point)) { // a sign must be followed by an integer or the dot
return report_parse_error<UC>(
p, parse_error::missing_integer_or_dot_after_sign);
}
}
}
UC const * const start_digits = p;
UC const *const start_digits = p;
uint64_t i = 0; // an unsigned int avoids signed overflows (which are bad)
@@ -307,24 +322,29 @@ parsed_number_string_t<UC> parse_number_string(UC const *p, UC const * pend, par
// a multiplication by 10 is cheaper than an arbitrary integer
// multiplication
i = 10 * i +
uint64_t(*p - UC('0')); // might overflow, we will handle the overflow later
uint64_t(*p -
UC('0')); // might overflow, we will handle the overflow later
++p;
}
UC const * const end_of_integer_part = p;
UC const *const end_of_integer_part = p;
int64_t digit_count = int64_t(end_of_integer_part - start_digits);
answer.integer = span<const UC>(start_digits, size_t(digit_count));
if (fmt & FASTFLOAT_JSONFMT) {
answer.integer = span<UC const>(start_digits, size_t(digit_count));
FASTFLOAT_IF_CONSTEXPR17(basic_json_fmt) {
// at least 1 digit in integer part, without leading zeros
if (digit_count == 0 || (start_digits[0] == UC('0') && digit_count > 1)) {
return answer;
if (digit_count == 0) {
return report_parse_error<UC>(p, parse_error::no_digits_in_integer_part);
}
if ((start_digits[0] == UC('0') && digit_count > 1)) {
return report_parse_error<UC>(start_digits,
parse_error::leading_zeros_in_integer_part);
}
}
int64_t exponent = 0;
const bool has_decimal_point = (p != pend) && (*p == decimal_point);
bool const has_decimal_point = (p != pend) && (*p == decimal_point);
if (has_decimal_point) {
++p;
UC const * before = p;
UC const *before = p;
// can occur at most twice without overflowing, but let it occur more, since
// for integers with many digits, digit parsing is the primary bottleneck.
loop_parse_if_eight_digits(p, pend, i);
@@ -335,41 +355,45 @@ parsed_number_string_t<UC> parse_number_string(UC const *p, UC const * pend, par
i = i * 10 + digit; // in rare cases, this will overflow, but that's ok
}
exponent = before - p;
answer.fraction = span<const UC>(before, size_t(p - before));
answer.fraction = span<UC const>(before, size_t(p - before));
digit_count -= exponent;
}
if (fmt & FASTFLOAT_JSONFMT) {
FASTFLOAT_IF_CONSTEXPR17(basic_json_fmt) {
// at least 1 digit in fractional part
if (has_decimal_point && exponent == 0) {
return answer;
return report_parse_error<UC>(p,
parse_error::no_digits_in_fractional_part);
}
}
else if (digit_count == 0) { // we must have encountered at least one integer!
return answer;
}
int64_t exp_number = 0; // explicit exponential part
if ( ((fmt & chars_format::scientific) &&
(p != pend) &&
((UC('e') == *p) || (UC('E') == *p)))
||
((fmt & FASTFLOAT_FORTRANFMT) &&
(p != pend) &&
((UC('+') == *p) || (UC('-') == *p) || (UC('d') == *p) || (UC('D') == *p)))) {
UC const * location_of_e = p;
if ((UC('e') == *p) || (UC('E') == *p) || (UC('d') == *p) || (UC('D') == *p)) {
else if (digit_count == 0) { // we must have encountered at least one integer!
return report_parse_error<UC>(p, parse_error::no_digits_in_mantissa);
}
int64_t exp_number = 0; // explicit exponential part
if ((uint64_t(fmt & chars_format::scientific) && (p != pend) &&
((UC('e') == *p) || (UC('E') == *p))) ||
(uint64_t(fmt & detail::basic_fortran_fmt) && (p != pend) &&
((UC('+') == *p) || (UC('-') == *p) || (UC('d') == *p) ||
(UC('D') == *p)))) {
UC const *location_of_e = p;
if ((UC('e') == *p) || (UC('E') == *p) || (UC('d') == *p) ||
(UC('D') == *p)) {
++p;
}
bool neg_exp = false;
if ((p != pend) && (UC('-') == *p)) {
neg_exp = true;
++p;
} else if ((p != pend) && (UC('+') == *p)) { // '+' on exponent is allowed by C++17 20.19.3.(7.1)
} else if ((p != pend) &&
(UC('+') ==
*p)) { // '+' on exponent is allowed by C++17 20.19.3.(7.1)
++p;
}
if ((p == pend) || !is_integer(*p)) {
if(!(fmt & chars_format::fixed)) {
// We are in error.
return answer;
if (!uint64_t(fmt & chars_format::fixed)) {
// The exponential part is invalid for scientific notation, so it must
// be a trailing token for fixed notation. However, fixed notation is
// disabled, so report a scientific notation error.
return report_parse_error<UC>(p, parse_error::missing_exponential_part);
}
// Otherwise, we will be ignoring the 'e'.
p = location_of_e;
@@ -381,12 +405,17 @@ parsed_number_string_t<UC> parse_number_string(UC const *p, UC const * pend, par
}
++p;
}
if(neg_exp) { exp_number = - exp_number; }
if (neg_exp) {
exp_number = -exp_number;
}
exponent += exp_number;
}
} else {
// If it scientific and not fixed, we have to bail out.
if((fmt & chars_format::scientific) && !(fmt & chars_format::fixed)) { return answer; }
if (uint64_t(fmt & chars_format::scientific) &&
!uint64_t(fmt & chars_format::fixed)) {
return report_parse_error<UC>(p, parse_error::missing_exponential_part);
}
}
answer.lastmatch = p;
answer.valid = true;
@@ -401,9 +430,11 @@ parsed_number_string_t<UC> parse_number_string(UC const *p, UC const * pend, par
// We have to handle the case where we have 0.0000somenumber.
// We need to be mindful of the case where we only have zeroes...
// E.g., 0.000000000...000.
UC const * start = start_digits;
UC const *start = start_digits;
while ((start != pend) && (*start == UC('0') || *start == decimal_point)) {
if(*start == UC('0')) { digit_count --; }
if (*start == UC('0')) {
digit_count--;
}
start++;
}
@@ -414,18 +445,17 @@ parsed_number_string_t<UC> parse_number_string(UC const *p, UC const * pend, par
// pre-tokenized spans from above.
i = 0;
p = answer.integer.ptr;
UC const* int_end = p + answer.integer.len();
const uint64_t minimal_nineteen_digit_integer{ 1000000000000000000 };
UC const *int_end = p + answer.integer.len();
uint64_t const minimal_nineteen_digit_integer{1000000000000000000};
while ((i < minimal_nineteen_digit_integer) && (p != int_end)) {
i = i * 10 + uint64_t(*p - UC('0'));
++p;
}
if (i >= minimal_nineteen_digit_integer) { // We have a big integers
exponent = end_of_integer_part - p + exp_number;
}
else { // We have a value with a fractional component.
} else { // We have a value with a fractional component.
p = answer.fraction.ptr;
UC const* frac_end = p + answer.fraction.len();
UC const *frac_end = p + answer.fraction.len();
while ((i < minimal_nineteen_digit_integer) && (p != frac_end)) {
i = i * 10 + uint64_t(*p - UC('0'));
++p;
@@ -441,35 +471,43 @@ parsed_number_string_t<UC> parse_number_string(UC const *p, UC const * pend, par
}
template <typename T, typename UC>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
from_chars_result_t<UC> parse_int_string(UC const* p, UC const* pend, T& value, int base) {
from_chars_result_t<UC> answer;
UC const* const first = p;
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC>
parse_int_string(UC const *p, UC const *pend, T &value,
parse_options_t<UC> options) {
chars_format const fmt = detail::adjust_for_feature_macros(options.format);
int const base = options.base;
bool negative = (*p == UC('-'));
from_chars_result_t<UC> answer;
UC const *const first = p;
bool const negative = (*p == UC('-'));
#ifdef FASTFLOAT_VISUAL_STUDIO
#pragma warning(push)
#pragma warning(disable : 4127)
#endif
if (!std::is_signed<T>::value && negative) {
#ifdef FASTFLOAT_VISUAL_STUDIO
#pragma warning(pop)
#endif
answer.ec = std::errc::invalid_argument;
answer.ptr = first;
return answer;
}
#ifdef FASTFLOAT_ALLOWS_LEADING_PLUS // disabled by default
if ((*p == UC('-')) || (*p == UC('+'))) {
#else
if (*p == UC('-')) {
#endif
if ((*p == UC('-')) ||
(uint64_t(fmt & chars_format::allow_leading_plus) && (*p == UC('+')))) {
++p;
}
UC const* const start_num = p;
UC const *const start_num = p;
while (p!= pend && *p == UC('0')) {
++p;
while (p != pend && *p == UC('0')) {
++p;
}
const bool has_leading_zeros = p > start_num;
bool const has_leading_zeros = p > start_num;
UC const* const start_digits = p;
UC const *const start_digits = p;
uint64_t i = 0;
if (base == 10) {
@@ -481,9 +519,9 @@ from_chars_result_t<UC> parse_int_string(UC const* p, UC const* pend, T& value,
break;
}
i = uint64_t(base) * i + digit; // might overflow, check this later
p++;
p++;
}
size_t digit_count = size_t(p - start_digits);
if (digit_count == 0) {
@@ -491,12 +529,11 @@ from_chars_result_t<UC> parse_int_string(UC const* p, UC const* pend, T& value,
value = 0;
answer.ec = std::errc();
answer.ptr = p;
}
else {
} else {
answer.ec = std::errc::invalid_argument;
answer.ptr = first;
}
return answer;
return answer;
}
answer.ptr = p;
@@ -507,7 +544,8 @@ from_chars_result_t<UC> parse_int_string(UC const* p, UC const* pend, T& value,
answer.ec = std::errc::result_out_of_range;
return answer;
}
// this check can be eliminated for all other types, but they will all require a max_digits(base) equivalent
// this check can be eliminated for all other types, but they will all require
// a max_digits(base) equivalent
if (digit_count == max_digits && i < min_safe_u64(base)) {
answer.ec = std::errc::result_out_of_range;
return answer;
@@ -524,18 +562,22 @@ from_chars_result_t<UC> parse_int_string(UC const* p, UC const* pend, T& value,
if (negative) {
#ifdef FASTFLOAT_VISUAL_STUDIO
#pragma warning(push)
#pragma warning(disable: 4146)
#pragma warning(disable : 4146)
#endif
// this weird workaround is required because:
// - converting unsigned to signed when its value is greater than signed max is UB pre-C++23.
// - converting unsigned to signed when its value is greater than signed max
// is UB pre-C++23.
// - reinterpret_casting (~i + 1) would work, but it is not constexpr
// this is always optimized into a neg instruction (note: T is an integer type)
value = T(-std::numeric_limits<T>::max() - T(i - uint64_t(std::numeric_limits<T>::max())));
// this is always optimized into a neg instruction (note: T is an integer
// type)
value = T(-std::numeric_limits<T>::max() -
T(i - uint64_t(std::numeric_limits<T>::max())));
#ifdef FASTFLOAT_VISUAL_STUDIO
#pragma warning(pop)
#endif
} else {
value = T(i);
}
else { value = T(i); }
answer.ec = std::errc();
return answer;

View File

@@ -37,15 +37,14 @@ constexpr size_t bigint_limbs = bigint_bits / limb_bits;
// vector-like type that is allocated on the stack. the entire
// buffer is pre-allocated, and only the length changes.
template <uint16_t size>
struct stackvec {
template <uint16_t size> struct stackvec {
limb data[size];
// we never need more than 150 limbs
uint16_t length{0};
stackvec() = default;
stackvec(const stackvec &) = delete;
stackvec &operator=(const stackvec &) = delete;
stackvec(stackvec const &) = delete;
stackvec &operator=(stackvec const &) = delete;
stackvec(stackvec &&) = delete;
stackvec &operator=(stackvec &&other) = delete;
@@ -54,16 +53,18 @@ struct stackvec {
FASTFLOAT_ASSERT(try_extend(s));
}
FASTFLOAT_CONSTEXPR14 limb& operator[](size_t index) noexcept {
FASTFLOAT_CONSTEXPR14 limb &operator[](size_t index) noexcept {
FASTFLOAT_DEBUG_ASSERT(index < length);
return data[index];
}
FASTFLOAT_CONSTEXPR14 const limb& operator[](size_t index) const noexcept {
FASTFLOAT_CONSTEXPR14 const limb &operator[](size_t index) const noexcept {
FASTFLOAT_DEBUG_ASSERT(index < length);
return data[index];
}
// index from the end of the container
FASTFLOAT_CONSTEXPR14 const limb& rindex(size_t index) const noexcept {
FASTFLOAT_CONSTEXPR14 const limb &rindex(size_t index) const noexcept {
FASTFLOAT_DEBUG_ASSERT(index < length);
size_t rindex = length - index - 1;
return data[rindex];
@@ -73,20 +74,19 @@ struct stackvec {
FASTFLOAT_CONSTEXPR14 void set_len(size_t len) noexcept {
length = uint16_t(len);
}
constexpr size_t len() const noexcept {
return length;
}
constexpr bool is_empty() const noexcept {
return length == 0;
}
constexpr size_t capacity() const noexcept {
return size;
}
constexpr size_t len() const noexcept { return length; }
constexpr bool is_empty() const noexcept { return length == 0; }
constexpr size_t capacity() const noexcept { return size; }
// append item to vector, without bounds checking
FASTFLOAT_CONSTEXPR14 void push_unchecked(limb value) noexcept {
data[length] = value;
length++;
}
// append item to vector, returning if item was added
FASTFLOAT_CONSTEXPR14 bool try_push(limb value) noexcept {
if (len() < capacity()) {
@@ -96,12 +96,14 @@ struct stackvec {
return false;
}
}
// add items to the vector, from a span, without bounds checking
FASTFLOAT_CONSTEXPR20 void extend_unchecked(limb_span s) noexcept {
limb* ptr = data + length;
limb *ptr = data + length;
std::copy_n(s.ptr, s.len(), ptr);
set_len(len() + s.len());
}
// try to add items to the vector, returning if items were added
FASTFLOAT_CONSTEXPR20 bool try_extend(limb_span s) noexcept {
if (len() + s.len() <= capacity()) {
@@ -111,6 +113,7 @@ struct stackvec {
return false;
}
}
// resize the vector, without bounds checking
// if the new size is longer than the vector, assign value to each
// appended item.
@@ -118,14 +121,15 @@ struct stackvec {
void resize_unchecked(size_t new_len, limb value) noexcept {
if (new_len > len()) {
size_t count = new_len - len();
limb* first = data + len();
limb* last = first + count;
limb *first = data + len();
limb *last = first + count;
::std::fill(first, last, value);
set_len(new_len);
} else {
set_len(new_len);
}
}
// try to resize the vector, returning if the vector was resized.
FASTFLOAT_CONSTEXPR20 bool try_resize(size_t new_len, limb value) noexcept {
if (new_len > capacity()) {
@@ -135,6 +139,7 @@ struct stackvec {
return true;
}
}
// check if any limbs are non-zero after the given index.
// this needs to be done in reverse order, since the index
// is relative to the most significant limbs.
@@ -147,6 +152,7 @@ struct stackvec {
}
return false;
}
// normalize the big integer, so most-significant zero limbs are removed.
FASTFLOAT_CONSTEXPR14 void normalize() noexcept {
while (len() > 0 && rindex(0) == 0) {
@@ -155,21 +161,21 @@ struct stackvec {
}
};
fastfloat_really_inline FASTFLOAT_CONSTEXPR14
uint64_t empty_hi64(bool& truncated) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR14 uint64_t
empty_hi64(bool &truncated) noexcept {
truncated = false;
return 0;
}
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
uint64_t uint64_hi64(uint64_t r0, bool& truncated) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 uint64_t
uint64_hi64(uint64_t r0, bool &truncated) noexcept {
truncated = false;
int shl = leading_zeroes(r0);
return r0 << shl;
}
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
uint64_t uint64_hi64(uint64_t r0, uint64_t r1, bool& truncated) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 uint64_t
uint64_hi64(uint64_t r0, uint64_t r1, bool &truncated) noexcept {
int shl = leading_zeroes(r0);
if (shl == 0) {
truncated = r1 != 0;
@@ -181,20 +187,20 @@ uint64_t uint64_hi64(uint64_t r0, uint64_t r1, bool& truncated) noexcept {
}
}
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
uint64_t uint32_hi64(uint32_t r0, bool& truncated) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 uint64_t
uint32_hi64(uint32_t r0, bool &truncated) noexcept {
return uint64_hi64(r0, truncated);
}
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
uint64_t uint32_hi64(uint32_t r0, uint32_t r1, bool& truncated) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 uint64_t
uint32_hi64(uint32_t r0, uint32_t r1, bool &truncated) noexcept {
uint64_t x0 = r0;
uint64_t x1 = r1;
return uint64_hi64((x0 << 32) | x1, truncated);
}
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
uint64_t uint32_hi64(uint32_t r0, uint32_t r1, uint32_t r2, bool& truncated) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 uint64_t
uint32_hi64(uint32_t r0, uint32_t r1, uint32_t r2, bool &truncated) noexcept {
uint64_t x0 = r0;
uint64_t x1 = r1;
uint64_t x2 = r2;
@@ -205,17 +211,17 @@ uint64_t uint32_hi64(uint32_t r0, uint32_t r1, uint32_t r2, bool& truncated) noe
// we want an efficient operation. for msvc, where
// we don't have built-in intrinsics, this is still
// pretty fast.
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
limb scalar_add(limb x, limb y, bool& overflow) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 limb
scalar_add(limb x, limb y, bool &overflow) noexcept {
limb z;
// gcc and clang
#if defined(__has_builtin)
#if __has_builtin(__builtin_add_overflow)
if (!cpp20_and_in_constexpr()) {
overflow = __builtin_add_overflow(x, y, &z);
return z;
}
#endif
#if __has_builtin(__builtin_add_overflow)
if (!cpp20_and_in_constexpr()) {
overflow = __builtin_add_overflow(x, y, &z);
return z;
}
#endif
#endif
// generic, this still optimizes correctly on MSVC.
@@ -225,24 +231,24 @@ limb scalar_add(limb x, limb y, bool& overflow) noexcept {
}
// multiply two small integers, getting both the high and low bits.
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
limb scalar_mul(limb x, limb y, limb& carry) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 limb
scalar_mul(limb x, limb y, limb &carry) noexcept {
#ifdef FASTFLOAT_64BIT_LIMB
#if defined(__SIZEOF_INT128__)
#if defined(__SIZEOF_INT128__)
// GCC and clang both define it as an extension.
__uint128_t z = __uint128_t(x) * __uint128_t(y) + __uint128_t(carry);
carry = limb(z >> limb_bits);
return limb(z);
#else
#else
// fallback, no native 128-bit integer multiplication with carry.
// on msvc, this optimizes identically, somehow.
value128 z = full_multiplication(x, y);
bool overflow;
z.low = scalar_add(z.low, carry, overflow);
z.high += uint64_t(overflow); // cannot overflow
z.high += uint64_t(overflow); // cannot overflow
carry = z.high;
return z.low;
#endif
#endif
#else
uint64_t z = uint64_t(x) * uint64_t(y) + uint64_t(carry);
carry = limb(z >> limb_bits);
@@ -253,8 +259,8 @@ limb scalar_mul(limb x, limb y, limb& carry) noexcept {
// add scalar value to bigint starting from offset.
// used in grade school multiplication
template <uint16_t size>
inline FASTFLOAT_CONSTEXPR20
bool small_add_from(stackvec<size>& vec, limb y, size_t start) noexcept {
inline FASTFLOAT_CONSTEXPR20 bool small_add_from(stackvec<size> &vec, limb y,
size_t start) noexcept {
size_t index = start;
limb carry = y;
bool overflow;
@@ -271,15 +277,15 @@ bool small_add_from(stackvec<size>& vec, limb y, size_t start) noexcept {
// add scalar value to bigint.
template <uint16_t size>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
bool small_add(stackvec<size>& vec, limb y) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 bool
small_add(stackvec<size> &vec, limb y) noexcept {
return small_add_from(vec, y, 0);
}
// multiply bigint by scalar value.
template <uint16_t size>
inline FASTFLOAT_CONSTEXPR20
bool small_mul(stackvec<size>& vec, limb y) noexcept {
inline FASTFLOAT_CONSTEXPR20 bool small_mul(stackvec<size> &vec,
limb y) noexcept {
limb carry = 0;
for (size_t index = 0; index < vec.len(); index++) {
vec[index] = scalar_mul(vec[index], y, carry);
@@ -293,12 +299,12 @@ bool small_mul(stackvec<size>& vec, limb y) noexcept {
// add bigint to bigint starting from index.
// used in grade school multiplication
template <uint16_t size>
FASTFLOAT_CONSTEXPR20
bool large_add_from(stackvec<size>& x, limb_span y, size_t start) noexcept {
FASTFLOAT_CONSTEXPR20 bool large_add_from(stackvec<size> &x, limb_span y,
size_t start) noexcept {
// the effective x buffer is from `xstart..x.len()`, so exit early
// if we can't get that current range.
if (x.len() < start || y.len() > x.len() - start) {
FASTFLOAT_TRY(x.try_resize(y.len() + start, 0));
FASTFLOAT_TRY(x.try_resize(y.len() + start, 0));
}
bool carry = false;
@@ -324,15 +330,14 @@ bool large_add_from(stackvec<size>& x, limb_span y, size_t start) noexcept {
// add bigint to bigint.
template <uint16_t size>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
bool large_add_from(stackvec<size>& x, limb_span y) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 bool
large_add_from(stackvec<size> &x, limb_span y) noexcept {
return large_add_from(x, y, 0);
}
// grade-school multiplication algorithm
template <uint16_t size>
FASTFLOAT_CONSTEXPR20
bool long_mul(stackvec<size>& x, limb_span y) noexcept {
FASTFLOAT_CONSTEXPR20 bool long_mul(stackvec<size> &x, limb_span y) noexcept {
limb_span xs = limb_span(x.data, x.len());
stackvec<size> z(xs);
limb_span zs = limb_span(z.data, z.len());
@@ -360,8 +365,7 @@ bool long_mul(stackvec<size>& x, limb_span y) noexcept {
// grade-school multiplication algorithm
template <uint16_t size>
FASTFLOAT_CONSTEXPR20
bool large_mul(stackvec<size>& x, limb_span y) noexcept {
FASTFLOAT_CONSTEXPR20 bool large_mul(stackvec<size> &x, limb_span y) noexcept {
if (y.len() == 1) {
FASTFLOAT_TRY(small_mul(x, y[0]));
} else {
@@ -370,36 +374,58 @@ bool large_mul(stackvec<size>& x, limb_span y) noexcept {
return true;
}
template <typename = void>
struct pow5_tables {
template <typename = void> struct pow5_tables {
static constexpr uint32_t large_step = 135;
static constexpr uint64_t small_power_of_5[] = {
1UL, 5UL, 25UL, 125UL, 625UL, 3125UL, 15625UL, 78125UL, 390625UL,
1953125UL, 9765625UL, 48828125UL, 244140625UL, 1220703125UL,
6103515625UL, 30517578125UL, 152587890625UL, 762939453125UL,
3814697265625UL, 19073486328125UL, 95367431640625UL, 476837158203125UL,
2384185791015625UL, 11920928955078125UL, 59604644775390625UL,
298023223876953125UL, 1490116119384765625UL, 7450580596923828125UL,
1UL,
5UL,
25UL,
125UL,
625UL,
3125UL,
15625UL,
78125UL,
390625UL,
1953125UL,
9765625UL,
48828125UL,
244140625UL,
1220703125UL,
6103515625UL,
30517578125UL,
152587890625UL,
762939453125UL,
3814697265625UL,
19073486328125UL,
95367431640625UL,
476837158203125UL,
2384185791015625UL,
11920928955078125UL,
59604644775390625UL,
298023223876953125UL,
1490116119384765625UL,
7450580596923828125UL,
};
#ifdef FASTFLOAT_64BIT_LIMB
constexpr static limb large_power_of_5[] = {
1414648277510068013UL, 9180637584431281687UL, 4539964771860779200UL,
10482974169319127550UL, 198276706040285095UL};
1414648277510068013UL, 9180637584431281687UL, 4539964771860779200UL,
10482974169319127550UL, 198276706040285095UL};
#else
constexpr static limb large_power_of_5[] = {
4279965485U, 329373468U, 4020270615U, 2137533757U, 4287402176U,
1057042919U, 1071430142U, 2440757623U, 381945767U, 46164893U};
4279965485U, 329373468U, 4020270615U, 2137533757U, 4287402176U,
1057042919U, 1071430142U, 2440757623U, 381945767U, 46164893U};
#endif
};
template <typename T>
constexpr uint32_t pow5_tables<T>::large_step;
#if FASTFLOAT_DETAIL_MUST_DEFINE_CONSTEXPR_VARIABLE
template <typename T>
constexpr uint64_t pow5_tables<T>::small_power_of_5[];
template <typename T> constexpr uint32_t pow5_tables<T>::large_step;
template <typename T>
constexpr limb pow5_tables<T>::large_power_of_5[];
template <typename T> constexpr uint64_t pow5_tables<T>::small_power_of_5[];
template <typename T> constexpr limb pow5_tables<T>::large_power_of_5[];
#endif
// big integer type. implements a small subset of big integer
// arithmetic, using simple algorithms since asymptotically
@@ -409,13 +435,14 @@ struct bigint : pow5_tables<> {
// storage of the limbs, in little-endian order.
stackvec<bigint_limbs> vec;
FASTFLOAT_CONSTEXPR20 bigint(): vec() {}
bigint(const bigint &) = delete;
bigint &operator=(const bigint &) = delete;
FASTFLOAT_CONSTEXPR20 bigint() : vec() {}
bigint(bigint const &) = delete;
bigint &operator=(bigint const &) = delete;
bigint(bigint &&) = delete;
bigint &operator=(bigint &&other) = delete;
FASTFLOAT_CONSTEXPR20 bigint(uint64_t value): vec() {
FASTFLOAT_CONSTEXPR20 bigint(uint64_t value) : vec() {
#ifdef FASTFLOAT_64BIT_LIMB
vec.push_unchecked(value);
#else
@@ -427,7 +454,7 @@ struct bigint : pow5_tables<> {
// get the high 64 bits from the vector, and if bits were truncated.
// this is to get the significant digits for the float.
FASTFLOAT_CONSTEXPR20 uint64_t hi64(bool& truncated) const noexcept {
FASTFLOAT_CONSTEXPR20 uint64_t hi64(bool &truncated) const noexcept {
#ifdef FASTFLOAT_64BIT_LIMB
if (vec.len() == 0) {
return empty_hi64(truncated);
@@ -446,7 +473,8 @@ struct bigint : pow5_tables<> {
} else if (vec.len() == 2) {
return uint32_hi64(vec.rindex(0), vec.rindex(1), truncated);
} else {
uint64_t result = uint32_hi64(vec.rindex(0), vec.rindex(1), vec.rindex(2), truncated);
uint64_t result =
uint32_hi64(vec.rindex(0), vec.rindex(1), vec.rindex(2), truncated);
truncated |= vec.nonzero(3);
return result;
}
@@ -459,7 +487,7 @@ struct bigint : pow5_tables<> {
// positive, this is larger, otherwise they are equal.
// the limbs are stored in little-endian order, so we
// must compare the limbs in ever order.
FASTFLOAT_CONSTEXPR20 int compare(const bigint& other) const noexcept {
FASTFLOAT_CONSTEXPR20 int compare(bigint const &other) const noexcept {
if (vec.len() > other.vec.len()) {
return 1;
} else if (vec.len() < other.vec.len()) {
@@ -512,12 +540,12 @@ struct bigint : pow5_tables<> {
return false;
} else if (!vec.is_empty()) {
// move limbs
limb* dst = vec.data + n;
const limb* src = vec.data;
limb *dst = vec.data + n;
limb const *src = vec.data;
std::copy_backward(src, src + vec.len(), dst + vec.len());
// fill in empty limbs
limb* first = vec.data;
limb* last = first + n;
limb *first = vec.data;
limb *last = first + n;
::std::fill(first, last, 0);
vec.set_len(n + vec.len());
return true;
@@ -560,18 +588,12 @@ struct bigint : pow5_tables<> {
return int(limb_bits * vec.len()) - lz;
}
FASTFLOAT_CONSTEXPR20 bool mul(limb y) noexcept {
return small_mul(vec, y);
}
FASTFLOAT_CONSTEXPR20 bool mul(limb y) noexcept { return small_mul(vec, y); }
FASTFLOAT_CONSTEXPR20 bool add(limb y) noexcept {
return small_add(vec, y);
}
FASTFLOAT_CONSTEXPR20 bool add(limb y) noexcept { return small_add(vec, y); }
// multiply as if by 2 raised to a power.
FASTFLOAT_CONSTEXPR20 bool pow2(uint32_t exp) noexcept {
return shl(exp);
}
FASTFLOAT_CONSTEXPR20 bool pow2(uint32_t exp) noexcept { return shl(exp); }
// multiply as if by 5 raised to a power.
FASTFLOAT_CONSTEXPR20 bool pow5(uint32_t exp) noexcept {
@@ -597,9 +619,8 @@ struct bigint : pow5_tables<> {
// Work around clang bug https://godbolt.org/z/zedh7rrhc
// This is similar to https://github.com/llvm/llvm-project/issues/47746,
// except the workaround described there don't work here
FASTFLOAT_TRY(
small_mul(vec, limb(((void)small_power_of_5[0], small_power_of_5[exp])))
);
FASTFLOAT_TRY(small_mul(
vec, limb(((void)small_power_of_5[0], small_power_of_5[exp]))));
}
return true;

View File

@@ -8,7 +8,7 @@
#endif
// Testing for https://wg21.link/N3652, adopted in C++14
#if __cpp_constexpr >= 201304
#if defined(__cpp_constexpr) && __cpp_constexpr >= 201304
#define FASTFLOAT_CONSTEXPR14 constexpr
#else
#define FASTFLOAT_CONSTEXPR14
@@ -20,16 +20,23 @@
#define FASTFLOAT_HAS_BIT_CAST 0
#endif
#if defined(__cpp_lib_is_constant_evaluated) && __cpp_lib_is_constant_evaluated >= 201811L
#if defined(__cpp_lib_is_constant_evaluated) && \
__cpp_lib_is_constant_evaluated >= 201811L
#define FASTFLOAT_HAS_IS_CONSTANT_EVALUATED 1
#else
#define FASTFLOAT_HAS_IS_CONSTANT_EVALUATED 0
#endif
#if defined(__cpp_if_constexpr) && __cpp_if_constexpr >= 201606L
#define FASTFLOAT_IF_CONSTEXPR17(x) if constexpr (x)
#else
#define FASTFLOAT_IF_CONSTEXPR17(x) if (x)
#endif
// Testing for relevant C++20 constexpr library features
#if FASTFLOAT_HAS_IS_CONSTANT_EVALUATED \
&& FASTFLOAT_HAS_BIT_CAST \
&& __cpp_lib_constexpr_algorithms >= 201806L /*For std::copy and std::fill*/
#if FASTFLOAT_HAS_IS_CONSTANT_EVALUATED && FASTFLOAT_HAS_BIT_CAST && \
defined(__cpp_lib_constexpr_algorithms) && \
__cpp_lib_constexpr_algorithms >= 201806L /*For std::copy and std::fill*/
#define FASTFLOAT_CONSTEXPR20 constexpr
#define FASTFLOAT_IS_CONSTEXPR 1
#else
@@ -37,4 +44,10 @@
#define FASTFLOAT_IS_CONSTEXPR 0
#endif
#if __cplusplus >= 201703L || (defined(_MSVC_LANG) && _MSVC_LANG >= 201703L)
#define FASTFLOAT_DETAIL_MUST_DEFINE_CONSTEXPR_VARIABLE 0
#else
#define FASTFLOAT_DETAIL_MUST_DEFINE_CONSTEXPR_VARIABLE 1
#endif
#endif // FASTFLOAT_CONSTEXPR_FEATURE_DETECT_H

View File

@@ -12,27 +12,34 @@
namespace fast_float {
// This will compute or rather approximate w * 5**q and return a pair of 64-bit words approximating
// the result, with the "high" part corresponding to the most significant bits and the
// low part corresponding to the least significant bits.
// This will compute or rather approximate w * 5**q and return a pair of 64-bit
// words approximating the result, with the "high" part corresponding to the
// most significant bits and the low part corresponding to the least significant
// bits.
//
template <int bit_precision>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
value128 compute_product_approximation(int64_t q, uint64_t w) {
const int index = 2 * int(q - powers::smallest_power_of_five);
// For small values of q, e.g., q in [0,27], the answer is always exact because
// The line value128 firstproduct = full_multiplication(w, power_of_five_128[index]);
// gives the exact answer.
value128 firstproduct = full_multiplication(w, powers::power_of_five_128[index]);
static_assert((bit_precision >= 0) && (bit_precision <= 64), " precision should be in (0,64]");
constexpr uint64_t precision_mask = (bit_precision < 64) ?
(uint64_t(0xFFFFFFFFFFFFFFFF) >> bit_precision)
: uint64_t(0xFFFFFFFFFFFFFFFF);
if((firstproduct.high & precision_mask) == precision_mask) { // could further guard with (lower + w < lower)
// regarding the second product, we only need secondproduct.high, but our expectation is that the compiler will optimize this extra work away if needed.
value128 secondproduct = full_multiplication(w, powers::power_of_five_128[index + 1]);
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 value128
compute_product_approximation(int64_t q, uint64_t w) {
int const index = 2 * int(q - powers::smallest_power_of_five);
// For small values of q, e.g., q in [0,27], the answer is always exact
// because The line value128 firstproduct = full_multiplication(w,
// power_of_five_128[index]); gives the exact answer.
value128 firstproduct =
full_multiplication(w, powers::power_of_five_128[index]);
static_assert((bit_precision >= 0) && (bit_precision <= 64),
" precision should be in (0,64]");
constexpr uint64_t precision_mask =
(bit_precision < 64) ? (uint64_t(0xFFFFFFFFFFFFFFFF) >> bit_precision)
: uint64_t(0xFFFFFFFFFFFFFFFF);
if ((firstproduct.high & precision_mask) ==
precision_mask) { // could further guard with (lower + w < lower)
// regarding the second product, we only need secondproduct.high, but our
// expectation is that the compiler will optimize this extra work away if
// needed.
value128 secondproduct =
full_multiplication(w, powers::power_of_five_128[index + 1]);
firstproduct.low += secondproduct.high;
if(secondproduct.high > firstproduct.low) {
if (secondproduct.high > firstproduct.low) {
firstproduct.high++;
}
}
@@ -55,43 +62,45 @@ namespace detail {
* where
* p = log(5**-q)/log(2) = -q * log(5)/log(2)
*/
constexpr fastfloat_really_inline int32_t power(int32_t q) noexcept {
return (((152170 + 65536) * q) >> 16) + 63;
}
constexpr fastfloat_really_inline int32_t power(int32_t q) noexcept {
return (((152170 + 65536) * q) >> 16) + 63;
}
} // namespace detail
// create an adjusted mantissa, biased by the invalid power2
// for significant digits already multiplied by 10 ** q.
template <typename binary>
fastfloat_really_inline FASTFLOAT_CONSTEXPR14
adjusted_mantissa compute_error_scaled(int64_t q, uint64_t w, int lz) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR14 adjusted_mantissa
compute_error_scaled(int64_t q, uint64_t w, int lz) noexcept {
int hilz = int(w >> 63) ^ 1;
adjusted_mantissa answer;
answer.mantissa = w << hilz;
int bias = binary::mantissa_explicit_bits() - binary::minimum_exponent();
answer.power2 = int32_t(detail::power(int32_t(q)) + bias - hilz - lz - 62 + invalid_am_bias);
answer.power2 = int32_t(detail::power(int32_t(q)) + bias - hilz - lz - 62 +
invalid_am_bias);
return answer;
}
// w * 10 ** q, without rounding the representation up.
// the power2 in the exponent will be adjusted by invalid_am_bias.
template <typename binary>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
adjusted_mantissa compute_error(int64_t q, uint64_t w) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 adjusted_mantissa
compute_error(int64_t q, uint64_t w) noexcept {
int lz = leading_zeroes(w);
w <<= lz;
value128 product = compute_product_approximation<binary::mantissa_explicit_bits() + 3>(q, w);
value128 product =
compute_product_approximation<binary::mantissa_explicit_bits() + 3>(q, w);
return compute_error_scaled<binary>(q, product.high, lz);
}
// w * 10 ** q
// The returned value should be a valid ieee64 number that simply need to be packed.
// However, in some very rare cases, the computation will fail. In such cases, we
// return an adjusted_mantissa with a negative power of 2: the caller should recompute
// in such cases.
// Computers w * 10 ** q.
// The returned value should be a valid number that simply needs to be
// packed. However, in some very rare cases, the computation will fail. In such
// cases, we return an adjusted_mantissa with a negative power of 2: the caller
// should recompute in such cases.
template <typename binary>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
adjusted_mantissa compute_float(int64_t q, uint64_t w) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 adjusted_mantissa
compute_float(int64_t q, uint64_t w) noexcept {
adjusted_mantissa answer;
if ((w == 0) || (q < binary::smallest_power_of_ten())) {
answer.power2 = 0;
@@ -105,7 +114,8 @@ adjusted_mantissa compute_float(int64_t q, uint64_t w) noexcept {
answer.mantissa = 0;
return answer;
}
// At this point in time q is in [powers::smallest_power_of_five, powers::largest_power_of_five].
// At this point in time q is in [powers::smallest_power_of_five,
// powers::largest_power_of_five].
// We want the most significant bit of i to be 1. Shift if needed.
int lz = leading_zeroes(w);
@@ -114,26 +124,32 @@ adjusted_mantissa compute_float(int64_t q, uint64_t w) noexcept {
// The required precision is binary::mantissa_explicit_bits() + 3 because
// 1. We need the implicit bit
// 2. We need an extra bit for rounding purposes
// 3. We might lose a bit due to the "upperbit" routine (result too small, requiring a shift)
// 3. We might lose a bit due to the "upperbit" routine (result too small,
// requiring a shift)
value128 product = compute_product_approximation<binary::mantissa_explicit_bits() + 3>(q, w);
value128 product =
compute_product_approximation<binary::mantissa_explicit_bits() + 3>(q, w);
// The computed 'product' is always sufficient.
// Mathematical proof:
// Noble Mushtak and Daniel Lemire, Fast Number Parsing Without Fallback (to appear)
// See script/mushtak_lemire.py
// Noble Mushtak and Daniel Lemire, Fast Number Parsing Without Fallback (to
// appear) See script/mushtak_lemire.py
// The "compute_product_approximation" function can be slightly slower than a branchless approach:
// value128 product = compute_product(q, w);
// but in practice, we can win big with the compute_product_approximation if its additional branch
// is easily predicted. Which is best is data specific.
// The "compute_product_approximation" function can be slightly slower than a
// branchless approach: value128 product = compute_product(q, w); but in
// practice, we can win big with the compute_product_approximation if its
// additional branch is easily predicted. Which is best is data specific.
int upperbit = int(product.high >> 63);
int shift = upperbit + 64 - binary::mantissa_explicit_bits() - 3;
answer.mantissa = product.high >> (upperbit + 64 - binary::mantissa_explicit_bits() - 3);
answer.mantissa = product.high >> shift;
answer.power2 = int32_t(detail::power(int32_t(q)) + upperbit - lz - binary::minimum_exponent());
answer.power2 = int32_t(detail::power(int32_t(q)) + upperbit - lz -
binary::minimum_exponent());
if (answer.power2 <= 0) { // we have a subnormal?
// Here have that answer.power2 <= 0 so -answer.power2 >= 0
if(-answer.power2 + 1 >= 64) { // if we have more than 64 bits below the minimum exponent, you have a zero for sure.
if (-answer.power2 + 1 >=
64) { // if we have more than 64 bits below the minimum exponent, you
// have a zero for sure.
answer.power2 = 0;
answer.mantissa = 0;
// result should be zero
@@ -142,7 +158,8 @@ adjusted_mantissa compute_float(int64_t q, uint64_t w) noexcept {
// next line is safe because -answer.power2 + 1 < 64
answer.mantissa >>= -answer.power2 + 1;
// Thankfully, we can't have both "round-to-even" and subnormals because
// "round-to-even" only occurs for powers close to 0.
// "round-to-even" only occurs for powers close to 0 in the 32-bit and
// and 64-bit case (with no more than 19 digits).
answer.mantissa += (answer.mantissa & 1); // round up
answer.mantissa >>= 1;
// There is a weird scenario where we don't have a subnormal but just.
@@ -152,20 +169,26 @@ adjusted_mantissa compute_float(int64_t q, uint64_t w) noexcept {
// up 0x3fffffffffffff x 2^-1023-53 and once we do, we are no longer
// subnormal, but we can only know this after rounding.
// So we only declare a subnormal if we are smaller than the threshold.
answer.power2 = (answer.mantissa < (uint64_t(1) << binary::mantissa_explicit_bits())) ? 0 : 1;
answer.power2 =
(answer.mantissa < (uint64_t(1) << binary::mantissa_explicit_bits()))
? 0
: 1;
return answer;
}
// usually, we round *up*, but if we fall right in between and and we have an
// even basis, we need to round down
// We are only concerned with the cases where 5**q fits in single 64-bit word.
if ((product.low <= 1) && (q >= binary::min_exponent_round_to_even()) && (q <= binary::max_exponent_round_to_even()) &&
((answer.mantissa & 3) == 1) ) { // we may fall between two floats!
if ((product.low <= 1) && (q >= binary::min_exponent_round_to_even()) &&
(q <= binary::max_exponent_round_to_even()) &&
((answer.mantissa & 3) == 1)) { // we may fall between two floats!
// To be in-between two floats we need that in doing
// answer.mantissa = product.high >> (upperbit + 64 - binary::mantissa_explicit_bits() - 3);
// ... we dropped out only zeroes. But if this happened, then we can go back!!!
if((answer.mantissa << (upperbit + 64 - binary::mantissa_explicit_bits() - 3)) == product.high) {
answer.mantissa &= ~uint64_t(1); // flip it so that we do not round up
// answer.mantissa = product.high >> (upperbit + 64 -
// binary::mantissa_explicit_bits() - 3);
// ... we dropped out only zeroes. But if this happened, then we can go
// back!!!
if ((answer.mantissa << shift) == product.high) {
answer.mantissa &= ~uint64_t(1); // flip it so that we do not round up
}
}

View File

@@ -13,19 +13,34 @@
namespace fast_float {
// 1e0 to 1e19
constexpr static uint64_t powers_of_ten_uint64[] = {
1UL, 10UL, 100UL, 1000UL, 10000UL, 100000UL, 1000000UL, 10000000UL, 100000000UL,
1000000000UL, 10000000000UL, 100000000000UL, 1000000000000UL, 10000000000000UL,
100000000000000UL, 1000000000000000UL, 10000000000000000UL, 100000000000000000UL,
1000000000000000000UL, 10000000000000000000UL};
constexpr static uint64_t powers_of_ten_uint64[] = {1UL,
10UL,
100UL,
1000UL,
10000UL,
100000UL,
1000000UL,
10000000UL,
100000000UL,
1000000000UL,
10000000000UL,
100000000000UL,
1000000000000UL,
10000000000000UL,
100000000000000UL,
1000000000000000UL,
10000000000000000UL,
100000000000000000UL,
1000000000000000000UL,
10000000000000000000UL};
// calculate the exponent, in scientific notation, of the number.
// this algorithm is not even close to optimized, but it has no practical
// effect on performance: in order to have a faster algorithm, we'd need
// to slow down performance for faster algorithms, and this is still fast.
template <typename UC>
fastfloat_really_inline FASTFLOAT_CONSTEXPR14
int32_t scientific_exponent(parsed_number_string_t<UC> & num) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR14 int32_t
scientific_exponent(parsed_number_string_t<UC> &num) noexcept {
uint64_t mantissa = num.mantissa;
int32_t exponent = int32_t(num.exponent);
while (mantissa >= 10000) {
@@ -45,15 +60,16 @@ int32_t scientific_exponent(parsed_number_string_t<UC> & num) noexcept {
// this converts a native floating-point number to an extended-precision float.
template <typename T>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
adjusted_mantissa to_extended(T value) noexcept {
using equiv_uint = typename binary_format<T>::equiv_uint;
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 adjusted_mantissa
to_extended(T value) noexcept {
using equiv_uint = equiv_uint_t<T>;
constexpr equiv_uint exponent_mask = binary_format<T>::exponent_mask();
constexpr equiv_uint mantissa_mask = binary_format<T>::mantissa_mask();
constexpr equiv_uint hidden_bit_mask = binary_format<T>::hidden_bit_mask();
adjusted_mantissa am;
int32_t bias = binary_format<T>::mantissa_explicit_bits() - binary_format<T>::minimum_exponent();
int32_t bias = binary_format<T>::mantissa_explicit_bits() -
binary_format<T>::minimum_exponent();
equiv_uint bits;
#if FASTFLOAT_HAS_BIT_CAST
bits = std::bit_cast<equiv_uint>(value);
@@ -66,7 +82,8 @@ adjusted_mantissa to_extended(T value) noexcept {
am.mantissa = bits & mantissa_mask;
} else {
// normal
am.power2 = int32_t((bits & exponent_mask) >> binary_format<T>::mantissa_explicit_bits());
am.power2 = int32_t((bits & exponent_mask) >>
binary_format<T>::mantissa_explicit_bits());
am.power2 -= bias;
am.mantissa = (bits & mantissa_mask) | hidden_bit_mask;
}
@@ -78,8 +95,8 @@ adjusted_mantissa to_extended(T value) noexcept {
// we are given a native float that represents b, so we need to adjust it
// halfway between b and b+u.
template <typename T>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
adjusted_mantissa to_extended_halfway(T value) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 adjusted_mantissa
to_extended_halfway(T value) noexcept {
adjusted_mantissa am = to_extended(value);
am.mantissa <<= 1;
am.mantissa += 1;
@@ -89,15 +106,18 @@ adjusted_mantissa to_extended_halfway(T value) noexcept {
// round an extended-precision float to the nearest machine float.
template <typename T, typename callback>
fastfloat_really_inline FASTFLOAT_CONSTEXPR14
void round(adjusted_mantissa& am, callback cb) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR14 void round(adjusted_mantissa &am,
callback cb) noexcept {
int32_t mantissa_shift = 64 - binary_format<T>::mantissa_explicit_bits() - 1;
if (-am.power2 >= mantissa_shift) {
// have a denormal float
int32_t shift = -am.power2 + 1;
cb(am, std::min<int32_t>(shift, 64));
// check for round-up: if rounding-nearest carried us to the hidden bit.
am.power2 = (am.mantissa < (uint64_t(1) << binary_format<T>::mantissa_explicit_bits())) ? 0 : 1;
am.power2 = (am.mantissa <
(uint64_t(1) << binary_format<T>::mantissa_explicit_bits()))
? 0
: 1;
return;
}
@@ -105,7 +125,8 @@ void round(adjusted_mantissa& am, callback cb) noexcept {
cb(am, mantissa_shift);
// check for carry
if (am.mantissa >= (uint64_t(2) << binary_format<T>::mantissa_explicit_bits())) {
if (am.mantissa >=
(uint64_t(2) << binary_format<T>::mantissa_explicit_bits())) {
am.mantissa = (uint64_t(1) << binary_format<T>::mantissa_explicit_bits());
am.power2++;
}
@@ -119,16 +140,11 @@ void round(adjusted_mantissa& am, callback cb) noexcept {
}
template <typename callback>
fastfloat_really_inline FASTFLOAT_CONSTEXPR14
void round_nearest_tie_even(adjusted_mantissa& am, int32_t shift, callback cb) noexcept {
const uint64_t mask
= (shift == 64)
? UINT64_MAX
: (uint64_t(1) << shift) - 1;
const uint64_t halfway
= (shift == 0)
? 0
: uint64_t(1) << (shift - 1);
fastfloat_really_inline FASTFLOAT_CONSTEXPR14 void
round_nearest_tie_even(adjusted_mantissa &am, int32_t shift,
callback cb) noexcept {
uint64_t const mask = (shift == 64) ? UINT64_MAX : (uint64_t(1) << shift) - 1;
uint64_t const halfway = (shift == 0) ? 0 : uint64_t(1) << (shift - 1);
uint64_t truncated_bits = am.mantissa & mask;
bool is_above = truncated_bits > halfway;
bool is_halfway = truncated_bits == halfway;
@@ -145,8 +161,8 @@ void round_nearest_tie_even(adjusted_mantissa& am, int32_t shift, callback cb) n
am.mantissa += uint64_t(cb(is_odd, is_halfway, is_above));
}
fastfloat_really_inline FASTFLOAT_CONSTEXPR14
void round_down(adjusted_mantissa& am, int32_t shift) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR14 void
round_down(adjusted_mantissa &am, int32_t shift) noexcept {
if (shift == 64) {
am.mantissa = 0;
} else {
@@ -154,11 +170,13 @@ void round_down(adjusted_mantissa& am, int32_t shift) noexcept {
}
am.power2 += shift;
}
template <typename UC>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
void skip_zeros(UC const * & first, UC const * last) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 void
skip_zeros(UC const *&first, UC const *last) noexcept {
uint64_t val;
while (!cpp20_and_in_constexpr() && std::distance(first, last) >= int_cmp_len<UC>()) {
while (!cpp20_and_in_constexpr() &&
std::distance(first, last) >= int_cmp_len<UC>()) {
::memcpy(&val, first, sizeof(uint64_t));
if (val != int_cmp_zeros<UC>()) {
break;
@@ -176,11 +194,12 @@ void skip_zeros(UC const * & first, UC const * last) noexcept {
// determine if any non-zero digits were truncated.
// all characters must be valid digits.
template <typename UC>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
bool is_truncated(UC const * first, UC const * last) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 bool
is_truncated(UC const *first, UC const *last) noexcept {
// do 8-bit optimizations, can just compare to 8 literal 0s.
uint64_t val;
while (!cpp20_and_in_constexpr() && std::distance(first, last) >= int_cmp_len<UC>()) {
while (!cpp20_and_in_constexpr() &&
std::distance(first, last) >= int_cmp_len<UC>()) {
::memcpy(&val, first, sizeof(uint64_t));
if (val != int_cmp_zeros<UC>()) {
return true;
@@ -195,16 +214,17 @@ bool is_truncated(UC const * first, UC const * last) noexcept {
}
return false;
}
template <typename UC>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
bool is_truncated(span<const UC> s) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 bool
is_truncated(span<UC const> s) noexcept {
return is_truncated(s.ptr, s.ptr + s.len());
}
template <typename UC>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
void parse_eight_digits(const UC*& p, limb& value, size_t& counter, size_t& count) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 void
parse_eight_digits(UC const *&p, limb &value, size_t &counter,
size_t &count) noexcept {
value = value * 100000000 + parse_eight_digits_unrolled(p);
p += 8;
counter += 8;
@@ -212,22 +232,23 @@ void parse_eight_digits(const UC*& p, limb& value, size_t& counter, size_t& coun
}
template <typename UC>
fastfloat_really_inline FASTFLOAT_CONSTEXPR14
void parse_one_digit(UC const *& p, limb& value, size_t& counter, size_t& count) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR14 void
parse_one_digit(UC const *&p, limb &value, size_t &counter,
size_t &count) noexcept {
value = value * 10 + limb(*p - UC('0'));
p++;
counter++;
count++;
}
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
void add_native(bigint& big, limb power, limb value) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 void
add_native(bigint &big, limb power, limb value) noexcept {
big.mul(power);
big.add(value);
}
fastfloat_really_inline FASTFLOAT_CONSTEXPR20
void round_up_bigint(bigint& big, size_t& count) noexcept {
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 void
round_up_bigint(bigint &big, size_t &count) noexcept {
// need to round-up the digits, but need to avoid rounding
// ....9999 to ...10000, which could cause a false halfway point.
add_native(big, 10, 1);
@@ -236,8 +257,9 @@ void round_up_bigint(bigint& big, size_t& count) noexcept {
// parse the significant digits into a big integer
template <typename UC>
inline FASTFLOAT_CONSTEXPR20
void parse_mantissa(bigint& result, parsed_number_string_t<UC>& num, size_t max_digits, size_t& digits) noexcept {
inline FASTFLOAT_CONSTEXPR20 void
parse_mantissa(bigint &result, parsed_number_string_t<UC> &num,
size_t max_digits, size_t &digits) noexcept {
// try to minimize the number of big integer and scalar multiplication.
// therefore, try to parse 8 digits at a time, and multiply by the largest
// scalar value (9 or 19 digits) for each step.
@@ -251,12 +273,13 @@ void parse_mantissa(bigint& result, parsed_number_string_t<UC>& num, size_t max_
#endif
// process all integer digits.
UC const * p = num.integer.ptr;
UC const * pend = p + num.integer.len();
UC const *p = num.integer.ptr;
UC const *pend = p + num.integer.len();
skip_zeros(p, pend);
// process all digits, in increments of step per loop
while (p != pend) {
while ((std::distance(p, pend) >= 8) && (step - counter >= 8) && (max_digits - digits >= 8)) {
while ((std::distance(p, pend) >= 8) && (step - counter >= 8) &&
(max_digits - digits >= 8)) {
parse_eight_digits(p, value, counter, digits);
}
while (counter < step && p != pend && digits < max_digits) {
@@ -289,7 +312,8 @@ void parse_mantissa(bigint& result, parsed_number_string_t<UC>& num, size_t max_
}
// process all digits, in increments of step per loop
while (p != pend) {
while ((std::distance(p, pend) >= 8) && (step - counter >= 8) && (max_digits - digits >= 8)) {
while ((std::distance(p, pend) >= 8) && (step - counter >= 8) &&
(max_digits - digits >= 8)) {
parse_eight_digits(p, value, counter, digits);
}
while (counter < step && p != pend && digits < max_digits) {
@@ -317,19 +341,23 @@ void parse_mantissa(bigint& result, parsed_number_string_t<UC>& num, size_t max_
}
template <typename T>
inline FASTFLOAT_CONSTEXPR20
adjusted_mantissa positive_digit_comp(bigint& bigmant, int32_t exponent) noexcept {
inline FASTFLOAT_CONSTEXPR20 adjusted_mantissa
positive_digit_comp(bigint &bigmant, int32_t exponent) noexcept {
FASTFLOAT_ASSERT(bigmant.pow10(uint32_t(exponent)));
adjusted_mantissa answer;
bool truncated;
answer.mantissa = bigmant.hi64(truncated);
int bias = binary_format<T>::mantissa_explicit_bits() - binary_format<T>::minimum_exponent();
int bias = binary_format<T>::mantissa_explicit_bits() -
binary_format<T>::minimum_exponent();
answer.power2 = bigmant.bit_length() - 64 + bias;
round<T>(answer, [truncated](adjusted_mantissa& a, int32_t shift) {
round_nearest_tie_even(a, shift, [truncated](bool is_odd, bool is_halfway, bool is_above) -> bool {
return is_above || (is_halfway && truncated) || (is_odd && is_halfway);
});
round<T>(answer, [truncated](adjusted_mantissa &a, int32_t shift) {
round_nearest_tie_even(
a, shift,
[truncated](bool is_odd, bool is_halfway, bool is_above) -> bool {
return is_above || (is_halfway && truncated) ||
(is_odd && is_halfway);
});
});
return answer;
@@ -341,15 +369,17 @@ adjusted_mantissa positive_digit_comp(bigint& bigmant, int32_t exponent) noexcep
// we then need to scale by `2^(f- e)`, and then the two significant digits
// are of the same magnitude.
template <typename T>
inline FASTFLOAT_CONSTEXPR20
adjusted_mantissa negative_digit_comp(bigint& bigmant, adjusted_mantissa am, int32_t exponent) noexcept {
bigint& real_digits = bigmant;
inline FASTFLOAT_CONSTEXPR20 adjusted_mantissa negative_digit_comp(
bigint &bigmant, adjusted_mantissa am, int32_t exponent) noexcept {
bigint &real_digits = bigmant;
int32_t real_exp = exponent;
// get the value of `b`, rounded down, and get a bigint representation of b+h
adjusted_mantissa am_b = am;
// gcc7 buf: use a lambda to remove the noexcept qualifier bug with -Wnoexcept-type.
round<T>(am_b, [](adjusted_mantissa&a, int32_t shift) { round_down(a, shift); });
// gcc7 buf: use a lambda to remove the noexcept qualifier bug with
// -Wnoexcept-type.
round<T>(am_b,
[](adjusted_mantissa &a, int32_t shift) { round_down(a, shift); });
T b;
to_float(false, am_b, b);
adjusted_mantissa theor = to_extended_halfway(b);
@@ -371,18 +401,19 @@ adjusted_mantissa negative_digit_comp(bigint& bigmant, adjusted_mantissa am, int
// compare digits, and use it to director rounding
int ord = real_digits.compare(theor_digits);
adjusted_mantissa answer = am;
round<T>(answer, [ord](adjusted_mantissa& a, int32_t shift) {
round_nearest_tie_even(a, shift, [ord](bool is_odd, bool _, bool __) -> bool {
(void)_; // not needed, since we've done our comparison
(void)__; // not needed, since we've done our comparison
if (ord > 0) {
return true;
} else if (ord < 0) {
return false;
} else {
return is_odd;
}
});
round<T>(answer, [ord](adjusted_mantissa &a, int32_t shift) {
round_nearest_tie_even(
a, shift, [ord](bool is_odd, bool _, bool __) -> bool {
(void)_; // not needed, since we've done our comparison
(void)__; // not needed, since we've done our comparison
if (ord > 0) {
return true;
} else if (ord < 0) {
return false;
} else {
return is_odd;
}
});
});
return answer;
@@ -402,8 +433,8 @@ adjusted_mantissa negative_digit_comp(bigint& bigmant, adjusted_mantissa am, int
// the actual digits. we then compare the big integer representations
// of both, and use that to direct rounding.
template <typename T, typename UC>
inline FASTFLOAT_CONSTEXPR20
adjusted_mantissa digit_comp(parsed_number_string_t<UC>& num, adjusted_mantissa am) noexcept {
inline FASTFLOAT_CONSTEXPR20 adjusted_mantissa
digit_comp(parsed_number_string_t<UC> &num, adjusted_mantissa am) noexcept {
// remove the invalid exponent bias
am.power2 -= invalid_am_bias;

View File

@@ -6,43 +6,54 @@
namespace fast_float {
/**
* This function parses the character sequence [first,last) for a number. It parses floating-point numbers expecting
* a locale-indepent format equivalent to what is used by std::strtod in the default ("C") locale.
* The resulting floating-point value is the closest floating-point values (using either float or double),
* using the "round to even" convention for values that would otherwise fall right in-between two values.
* That is, we provide exact parsing according to the IEEE standard.
* This function parses the character sequence [first,last) for a number. It
* parses floating-point numbers expecting a locale-indepent format equivalent
* to what is used by std::strtod in the default ("C") locale. The resulting
* floating-point value is the closest floating-point values (using either float
* or double), using the "round to even" convention for values that would
* otherwise fall right in-between two values. That is, we provide exact parsing
* according to the IEEE standard.
*
* Given a successful parse, the pointer (`ptr`) in the returned value is set to point right after the
* parsed number, and the `value` referenced is set to the parsed value. In case of error, the returned
* `ec` contains a representative error, otherwise the default (`std::errc()`) value is stored.
* Given a successful parse, the pointer (`ptr`) in the returned value is set to
* point right after the parsed number, and the `value` referenced is set to the
* parsed value. In case of error, the returned `ec` contains a representative
* error, otherwise the default (`std::errc()`) value is stored.
*
* The implementation does not throw and does not allocate memory (e.g., with `new` or `malloc`).
* The implementation does not throw and does not allocate memory (e.g., with
* `new` or `malloc`).
*
* Like the C++17 standard, the `fast_float::from_chars` functions take an optional last argument of
* the type `fast_float::chars_format`. It is a bitset value: we check whether
* `fmt & fast_float::chars_format::fixed` and `fmt & fast_float::chars_format::scientific` are set
* to determine whether we allow the fixed point and scientific notation respectively.
* The default is `fast_float::chars_format::general` which allows both `fixed` and `scientific`.
* Like the C++17 standard, the `fast_float::from_chars` functions take an
* optional last argument of the type `fast_float::chars_format`. It is a bitset
* value: we check whether `fmt & fast_float::chars_format::fixed` and `fmt &
* fast_float::chars_format::scientific` are set to determine whether we allow
* the fixed point and scientific notation respectively. The default is
* `fast_float::chars_format::general` which allows both `fixed` and
* `scientific`.
*/
template<typename T, typename UC = char, typename = FASTFLOAT_ENABLE_IF(is_supported_float_type<T>())>
FASTFLOAT_CONSTEXPR20
from_chars_result_t<UC> from_chars(UC const * first, UC const * last,
T &value, chars_format fmt = chars_format::general) noexcept;
template <typename T, typename UC = char,
typename = FASTFLOAT_ENABLE_IF(is_supported_float_type<T>::value)>
FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC>
from_chars(UC const *first, UC const *last, T &value,
chars_format fmt = chars_format::general) noexcept;
/**
* Like from_chars, but accepts an `options` argument to govern number parsing.
* Both for floating-point types and integer types.
*/
template<typename T, typename UC = char>
FASTFLOAT_CONSTEXPR20
from_chars_result_t<UC> from_chars_advanced(UC const * first, UC const * last,
T &value, parse_options_t<UC> options) noexcept;
template <typename T, typename UC = char>
FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC>
from_chars_advanced(UC const *first, UC const *last, T &value,
parse_options_t<UC> options) noexcept;
/**
* from_chars for integer types.
*/
template <typename T, typename UC = char, typename = FASTFLOAT_ENABLE_IF(!is_supported_float_type<T>())>
FASTFLOAT_CONSTEXPR20
from_chars_result_t<UC> from_chars(UC const * first, UC const * last, T& value, int base = 10) noexcept;
* from_chars for integer types.
*/
template <typename T, typename UC = char,
typename = FASTFLOAT_ENABLE_IF(is_supported_integer_type<T>::value)>
FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC>
from_chars(UC const *first, UC const *last, T &value, int base = 10) noexcept;
} // namespace fast_float
#include "parse_number.h"
#endif // FASTFLOAT_FAST_FLOAT_H

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -10,8 +10,8 @@
#include <cstring>
#include <limits>
#include <system_error>
namespace fast_float {
namespace fast_float {
namespace detail {
/**
@@ -20,45 +20,49 @@ namespace detail {
* strings a null-free and fixed.
**/
template <typename T, typename UC>
from_chars_result_t<UC> FASTFLOAT_CONSTEXPR14
parse_infnan(UC const * first, UC const * last, T &value) noexcept {
from_chars_result_t<UC>
FASTFLOAT_CONSTEXPR14 parse_infnan(UC const *first, UC const *last,
T &value, chars_format fmt) noexcept {
from_chars_result_t<UC> answer{};
answer.ptr = first;
answer.ec = std::errc(); // be optimistic
bool minusSign = false;
if (*first == UC('-')) { // assume first < last, so dereference without checks; C++17 20.19.3.(7.1) explicitly forbids '+' here
minusSign = true;
++first;
// assume first < last, so dereference without checks;
bool const minusSign = (*first == UC('-'));
// C++17 20.19.3.(7.1) explicitly forbids '+' sign here
if ((*first == UC('-')) ||
(uint64_t(fmt & chars_format::allow_leading_plus) &&
(*first == UC('+')))) {
++first;
}
#ifdef FASTFLOAT_ALLOWS_LEADING_PLUS // disabled by default
if (*first == UC('+')) {
++first;
}
#endif
if (last - first >= 3) {
if (fastfloat_strncasecmp(first, str_const_nan<UC>(), 3)) {
answer.ptr = (first += 3);
value = minusSign ? -std::numeric_limits<T>::quiet_NaN() : std::numeric_limits<T>::quiet_NaN();
// Check for possible nan(n-char-seq-opt), C++17 20.19.3.7, C11 7.20.1.3.3. At least MSVC produces nan(ind) and nan(snan).
if(first != last && *first == UC('(')) {
for(UC const * ptr = first + 1; ptr != last; ++ptr) {
value = minusSign ? -std::numeric_limits<T>::quiet_NaN()
: std::numeric_limits<T>::quiet_NaN();
// Check for possible nan(n-char-seq-opt), C++17 20.19.3.7,
// C11 7.20.1.3.3. At least MSVC produces nan(ind) and nan(snan).
if (first != last && *first == UC('(')) {
for (UC const *ptr = first + 1; ptr != last; ++ptr) {
if (*ptr == UC(')')) {
answer.ptr = ptr + 1; // valid nan(n-char-seq-opt)
break;
}
else if(!((UC('a') <= *ptr && *ptr <= UC('z')) || (UC('A') <= *ptr && *ptr <= UC('Z')) || (UC('0') <= *ptr && *ptr <= UC('9')) || *ptr == UC('_')))
} else if (!((UC('a') <= *ptr && *ptr <= UC('z')) ||
(UC('A') <= *ptr && *ptr <= UC('Z')) ||
(UC('0') <= *ptr && *ptr <= UC('9')) || *ptr == UC('_')))
break; // forbidden char, not nan(n-char-seq-opt)
}
}
return answer;
}
if (fastfloat_strncasecmp(first, str_const_inf<UC>(), 3)) {
if ((last - first >= 8) && fastfloat_strncasecmp(first + 3, str_const_inf<UC>() + 3, 5)) {
if ((last - first >= 8) &&
fastfloat_strncasecmp(first + 3, str_const_inf<UC>() + 3, 5)) {
answer.ptr = first + 8;
} else {
answer.ptr = first + 3;
}
value = minusSign ? -std::numeric_limits<T>::infinity() : std::numeric_limits<T>::infinity();
value = minusSign ? -std::numeric_limits<T>::infinity()
: std::numeric_limits<T>::infinity();
return answer;
}
}
@@ -86,73 +90,71 @@ fastfloat_really_inline bool rounds_to_nearest() noexcept {
// However, it is expected to be much faster than the fegetround()
// function call.
//
// The volatile keywoard prevents the compiler from computing the function
// The volatile keyword prevents the compiler from computing the function
// at compile-time.
// There might be other ways to prevent compile-time optimizations (e.g., asm).
// The value does not need to be std::numeric_limits<float>::min(), any small
// value so that 1 + x should round to 1 would do (after accounting for excess
// precision, as in 387 instructions).
static volatile float fmin = std::numeric_limits<float>::min();
// There might be other ways to prevent compile-time optimizations (e.g.,
// asm). The value does not need to be std::numeric_limits<float>::min(), any
// small value so that 1 + x should round to 1 would do (after accounting for
// excess precision, as in 387 instructions).
static float volatile fmin = std::numeric_limits<float>::min();
float fmini = fmin; // we copy it so that it gets loaded at most once.
//
// Explanation:
// Only when fegetround() == FE_TONEAREST do we have that
// fmin + 1.0f == 1.0f - fmin.
//
// FE_UPWARD:
// fmin + 1.0f > 1
// 1.0f - fmin == 1
//
// FE_DOWNWARD or FE_TOWARDZERO:
// fmin + 1.0f == 1
// 1.0f - fmin < 1
//
// Note: This may fail to be accurate if fast-math has been
// enabled, as rounding conventions may not apply.
#ifdef FASTFLOAT_VISUAL_STUDIO
# pragma warning(push)
// todo: is there a VS warning?
// see https://stackoverflow.com/questions/46079446/is-there-a-warning-for-floating-point-equality-checking-in-visual-studio-2013
#elif defined(__clang__)
# pragma clang diagnostic push
# pragma clang diagnostic ignored "-Wfloat-equal"
#elif defined(__GNUC__)
# pragma GCC diagnostic push
# pragma GCC diagnostic ignored "-Wfloat-equal"
#endif
//
// Explanation:
// Only when fegetround() == FE_TONEAREST do we have that
// fmin + 1.0f == 1.0f - fmin.
//
// FE_UPWARD:
// fmin + 1.0f > 1
// 1.0f - fmin == 1
//
// FE_DOWNWARD or FE_TOWARDZERO:
// fmin + 1.0f == 1
// 1.0f - fmin < 1
//
// Note: This may fail to be accurate if fast-math has been
// enabled, as rounding conventions may not apply.
#ifdef FASTFLOAT_VISUAL_STUDIO
#pragma warning(push)
// todo: is there a VS warning?
// see
// https://stackoverflow.com/questions/46079446/is-there-a-warning-for-floating-point-equality-checking-in-visual-studio-2013
#elif defined(__clang__)
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wfloat-equal"
#elif defined(__GNUC__)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wfloat-equal"
#endif
return (fmini + 1.0f == 1.0f - fmini);
#ifdef FASTFLOAT_VISUAL_STUDIO
# pragma warning(pop)
#elif defined(__clang__)
# pragma clang diagnostic pop
#elif defined(__GNUC__)
# pragma GCC diagnostic pop
#endif
#ifdef FASTFLOAT_VISUAL_STUDIO
#pragma warning(pop)
#elif defined(__clang__)
#pragma clang diagnostic pop
#elif defined(__GNUC__)
#pragma GCC diagnostic pop
#endif
}
} // namespace detail
template <typename T>
struct from_chars_caller
{
template <typename T> struct from_chars_caller {
template <typename UC>
FASTFLOAT_CONSTEXPR20
static from_chars_result_t<UC> call(UC const * first, UC const * last,
T &value, parse_options_t<UC> options) noexcept {
FASTFLOAT_CONSTEXPR20 static from_chars_result_t<UC>
call(UC const *first, UC const *last, T &value,
parse_options_t<UC> options) noexcept {
return from_chars_advanced(first, last, value, options);
}
};
#if __STDCPP_FLOAT32_T__ == 1
template <>
struct from_chars_caller<std::float32_t>
{
#ifdef __STDCPP_FLOAT32_T__
template <> struct from_chars_caller<std::float32_t> {
template <typename UC>
FASTFLOAT_CONSTEXPR20
static from_chars_result_t<UC> call(UC const * first, UC const * last,
std::float32_t &value, parse_options_t<UC> options) noexcept{
// if std::float32_t is defined, and we are in C++23 mode; macro set for float32;
// set value to float due to equivalence between float and float32_t
FASTFLOAT_CONSTEXPR20 static from_chars_result_t<UC>
call(UC const *first, UC const *last, std::float32_t &value,
parse_options_t<UC> options) noexcept {
// if std::float32_t is defined, and we are in C++23 mode; macro set for
// float32; set value to float due to equivalence between float and
// float32_t
float val;
auto ret = from_chars_advanced(first, last, val, options);
value = val;
@@ -161,16 +163,15 @@ struct from_chars_caller<std::float32_t>
};
#endif
#if __STDCPP_FLOAT64_T__ == 1
template <>
struct from_chars_caller<std::float64_t>
{
#ifdef __STDCPP_FLOAT64_T__
template <> struct from_chars_caller<std::float64_t> {
template <typename UC>
FASTFLOAT_CONSTEXPR20
static from_chars_result_t<UC> call(UC const * first, UC const * last,
std::float64_t &value, parse_options_t<UC> options) noexcept{
// if std::float64_t is defined, and we are in C++23 mode; macro set for float64;
// set value as double due to equivalence between double and float64_t
FASTFLOAT_CONSTEXPR20 static from_chars_result_t<UC>
call(UC const *first, UC const *last, std::float64_t &value,
parse_options_t<UC> options) noexcept {
// if std::float64_t is defined, and we are in C++23 mode; macro set for
// float64; set value as double due to equivalence between double and
// float64_t
double val;
auto ret = from_chars_advanced(first, last, val, options);
value = val;
@@ -179,52 +180,40 @@ struct from_chars_caller<std::float64_t>
};
#endif
template<typename T, typename UC, typename>
FASTFLOAT_CONSTEXPR20
from_chars_result_t<UC> from_chars(UC const * first, UC const * last,
T &value, chars_format fmt /*= chars_format::general*/) noexcept {
return from_chars_caller<T>::call(first, last, value, parse_options_t<UC>(fmt));
template <typename T, typename UC, typename>
FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC>
from_chars(UC const *first, UC const *last, T &value,
chars_format fmt /*= chars_format::general*/) noexcept {
return from_chars_caller<T>::call(first, last, value,
parse_options_t<UC>(fmt));
}
template<typename T, typename UC>
FASTFLOAT_CONSTEXPR20
from_chars_result_t<UC> from_chars_advanced(UC const * first, UC const * last,
T &value, parse_options_t<UC> options) noexcept {
/**
* This function overload takes parsed_number_string_t structure that is created
* and populated either by from_chars_advanced function taking chars range and
* parsing options or other parsing custom function implemented by user.
*/
template <typename T, typename UC>
FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC>
from_chars_advanced(parsed_number_string_t<UC> &pns, T &value) noexcept {
static_assert (is_supported_float_type<T>(), "only some floating-point types are supported");
static_assert (is_supported_char_type<UC>(), "only char, wchar_t, char16_t and char32_t are supported");
static_assert(is_supported_float_type<T>::value,
"only some floating-point types are supported");
static_assert(is_supported_char_type<UC>::value,
"only char, wchar_t, char16_t and char32_t are supported");
from_chars_result_t<UC> answer;
#ifdef FASTFLOAT_SKIP_WHITE_SPACE // disabled by default
while ((first != last) && fast_float::is_space(uint8_t(*first))) {
first++;
}
#endif
if (first == last) {
answer.ec = std::errc::invalid_argument;
answer.ptr = first;
return answer;
}
parsed_number_string_t<UC> pns = parse_number_string<UC>(first, last, options);
if (!pns.valid) {
if (options.format & chars_format::no_infnan) {
answer.ec = std::errc::invalid_argument;
answer.ptr = first;
return answer;
} else {
return detail::parse_infnan(first, last, value);
}
}
answer.ec = std::errc(); // be optimistic
answer.ptr = pns.lastmatch;
// The implementation of the Clinger's fast path is convoluted because
// we want round-to-nearest in all cases, irrespective of the rounding mode
// selected on the thread.
// We proceed optimistically, assuming that detail::rounds_to_nearest() returns
// true.
if (binary_format<T>::min_exponent_fast_path() <= pns.exponent && pns.exponent <= binary_format<T>::max_exponent_fast_path() && !pns.too_many_digits) {
// We proceed optimistically, assuming that detail::rounds_to_nearest()
// returns true.
if (binary_format<T>::min_exponent_fast_path() <= pns.exponent &&
pns.exponent <= binary_format<T>::max_exponent_fast_path() &&
!pns.too_many_digits) {
// Unfortunately, the conventional Clinger's fast path is only possible
// when the system rounds to the nearest float.
//
@@ -232,68 +221,179 @@ from_chars_result_t<UC> from_chars_advanced(UC const * first, UC const * last,
// We could check it first (before the previous branch), but
// there might be performance advantages at having the check
// be last.
if(!cpp20_and_in_constexpr() && detail::rounds_to_nearest()) {
if (!cpp20_and_in_constexpr() && detail::rounds_to_nearest()) {
// We have that fegetround() == FE_TONEAREST.
// Next is Clinger's fast path.
if (pns.mantissa <=binary_format<T>::max_mantissa_fast_path()) {
if (pns.mantissa <= binary_format<T>::max_mantissa_fast_path()) {
value = T(pns.mantissa);
if (pns.exponent < 0) { value = value / binary_format<T>::exact_power_of_ten(-pns.exponent); }
else { value = value * binary_format<T>::exact_power_of_ten(pns.exponent); }
if (pns.negative) { value = -value; }
if (pns.exponent < 0) {
value = value / binary_format<T>::exact_power_of_ten(-pns.exponent);
} else {
value = value * binary_format<T>::exact_power_of_ten(pns.exponent);
}
if (pns.negative) {
value = -value;
}
return answer;
}
} else {
// We do not have that fegetround() == FE_TONEAREST.
// Next is a modified Clinger's fast path, inspired by Jakub Jelínek's proposal
if (pns.exponent >= 0 && pns.mantissa <=binary_format<T>::max_mantissa_fast_path(pns.exponent)) {
// Next is a modified Clinger's fast path, inspired by Jakub Jelínek's
// proposal
if (pns.exponent >= 0 &&
pns.mantissa <=
binary_format<T>::max_mantissa_fast_path(pns.exponent)) {
#if defined(__clang__) || defined(FASTFLOAT_32BIT)
// Clang may map 0 to -0.0 when fegetround() == FE_DOWNWARD
if(pns.mantissa == 0) {
if (pns.mantissa == 0) {
value = pns.negative ? T(-0.) : T(0.);
return answer;
}
#endif
value = T(pns.mantissa) * binary_format<T>::exact_power_of_ten(pns.exponent);
if (pns.negative) { value = -value; }
value = T(pns.mantissa) *
binary_format<T>::exact_power_of_ten(pns.exponent);
if (pns.negative) {
value = -value;
}
return answer;
}
}
}
adjusted_mantissa am = compute_float<binary_format<T>>(pns.exponent, pns.mantissa);
if(pns.too_many_digits && am.power2 >= 0) {
if(am != compute_float<binary_format<T>>(pns.exponent, pns.mantissa + 1)) {
adjusted_mantissa am =
compute_float<binary_format<T>>(pns.exponent, pns.mantissa);
if (pns.too_many_digits && am.power2 >= 0) {
if (am != compute_float<binary_format<T>>(pns.exponent, pns.mantissa + 1)) {
am = compute_error<binary_format<T>>(pns.exponent, pns.mantissa);
}
}
// If we called compute_float<binary_format<T>>(pns.exponent, pns.mantissa) and we have an invalid power (am.power2 < 0),
// then we need to go the long way around again. This is very uncommon.
if(am.power2 < 0) { am = digit_comp<T>(pns, am); }
// If we called compute_float<binary_format<T>>(pns.exponent, pns.mantissa)
// and we have an invalid power (am.power2 < 0), then we need to go the long
// way around again. This is very uncommon.
if (am.power2 < 0) {
am = digit_comp<T>(pns, am);
}
to_float(pns.negative, am, value);
// Test for over/underflow.
if ((pns.mantissa != 0 && am.mantissa == 0 && am.power2 == 0) || am.power2 == binary_format<T>::infinite_power()) {
if ((pns.mantissa != 0 && am.mantissa == 0 && am.power2 == 0) ||
am.power2 == binary_format<T>::infinite_power()) {
answer.ec = std::errc::result_out_of_range;
}
return answer;
}
template <typename T, typename UC>
FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC>
from_chars_float_advanced(UC const *first, UC const *last, T &value,
parse_options_t<UC> options) noexcept {
template <typename T, typename UC, typename>
FASTFLOAT_CONSTEXPR20
from_chars_result_t<UC> from_chars(UC const* first, UC const* last, T& value, int base) noexcept {
static_assert (is_supported_char_type<UC>(), "only char, wchar_t, char16_t and char32_t are supported");
static_assert(is_supported_float_type<T>::value,
"only some floating-point types are supported");
static_assert(is_supported_char_type<UC>::value,
"only char, wchar_t, char16_t and char32_t are supported");
chars_format const fmt = detail::adjust_for_feature_macros(options.format);
from_chars_result_t<UC> answer;
#ifdef FASTFLOAT_SKIP_WHITE_SPACE // disabled by default
while ((first != last) && fast_float::is_space(uint8_t(*first))) {
first++;
if (uint64_t(fmt & chars_format::skip_white_space)) {
while ((first != last) && fast_float::is_space(*first)) {
first++;
}
}
if (first == last) {
answer.ec = std::errc::invalid_argument;
answer.ptr = first;
return answer;
}
parsed_number_string_t<UC> pns =
uint64_t(fmt & detail::basic_json_fmt)
? parse_number_string<true, UC>(first, last, options)
: parse_number_string<false, UC>(first, last, options);
if (!pns.valid) {
if (uint64_t(fmt & chars_format::no_infnan)) {
answer.ec = std::errc::invalid_argument;
answer.ptr = first;
return answer;
} else {
return detail::parse_infnan(first, last, value, fmt);
}
}
// call overload that takes parsed_number_string_t directly.
return from_chars_advanced(pns, value);
}
template <typename T, typename UC, typename>
FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC>
from_chars(UC const *first, UC const *last, T &value, int base) noexcept {
static_assert(is_supported_integer_type<T>::value,
"only integer types are supported");
static_assert(is_supported_char_type<UC>::value,
"only char, wchar_t, char16_t and char32_t are supported");
parse_options_t<UC> options;
options.base = base;
return from_chars_advanced(first, last, value, options);
}
template <typename T, typename UC>
FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC>
from_chars_int_advanced(UC const *first, UC const *last, T &value,
parse_options_t<UC> options) noexcept {
static_assert(is_supported_integer_type<T>::value,
"only integer types are supported");
static_assert(is_supported_char_type<UC>::value,
"only char, wchar_t, char16_t and char32_t are supported");
chars_format const fmt = detail::adjust_for_feature_macros(options.format);
int const base = options.base;
from_chars_result_t<UC> answer;
if (uint64_t(fmt & chars_format::skip_white_space)) {
while ((first != last) && fast_float::is_space(*first)) {
first++;
}
}
#endif
if (first == last || base < 2 || base > 36) {
answer.ec = std::errc::invalid_argument;
answer.ptr = first;
return answer;
}
return parse_int_string(first, last, value, base);
return parse_int_string(first, last, value, options);
}
template <size_t TypeIx> struct from_chars_advanced_caller {
static_assert(TypeIx > 0, "unsupported type");
};
template <> struct from_chars_advanced_caller<1> {
template <typename T, typename UC>
FASTFLOAT_CONSTEXPR20 static from_chars_result_t<UC>
call(UC const *first, UC const *last, T &value,
parse_options_t<UC> options) noexcept {
return from_chars_float_advanced(first, last, value, options);
}
};
template <> struct from_chars_advanced_caller<2> {
template <typename T, typename UC>
FASTFLOAT_CONSTEXPR20 static from_chars_result_t<UC>
call(UC const *first, UC const *last, T &value,
parse_options_t<UC> options) noexcept {
return from_chars_int_advanced(first, last, value, options);
}
};
template <typename T, typename UC>
FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC>
from_chars_advanced(UC const *first, UC const *last, T &value,
parse_options_t<UC> options) noexcept {
return from_chars_advanced_caller<
size_t(is_supported_float_type<T>::value) +
2 * size_t(is_supported_integer_type<T>::value)>::call(first, last, value,
options);
}
} // namespace fast_float

View File

@@ -1,3 +1,162 @@
# 11.2.0 - 2025-05-03
- Added the `s` specifier for `std::error_code`. It allows formatting an error
message as a string. For example:
```c++
#include <fmt/std.h>
int main() {
auto ec = std::make_error_code(std::errc::no_such_file_or_directory);
fmt::print("{:s}\n", ec);
}
```
prints
```
No such file or directory
```
(The actual message is platform-specific.)
- Fixed formatting of `std::chrono::local_time` and `tm`
(https://github.com/fmtlib/fmt/issues/3815,
https://github.com/fmtlib/fmt/issues/4350).
For example ([godbolt](https://www.godbolt.org/z/8o4b1PPn5)):
```c++
#include <fmt/chrono.h>
int main() {
std::chrono::zoned_time zt(
std::chrono::current_zone(),
std::chrono::system_clock::now());
fmt::print("{}", zt.get_local_time());
}
```
is now formatted consistenly across platforms.
- Added diagnostics for cases when timezone information is not available.
For example:
```c++
fmt::print("{:Z}", std::chrono::local_seconds());
```
now gives a compile-time error.
- Deprecated `fmt::localtime` in favor of `std::localtime`.
- Fixed compilation with GCC 15 and C++20 modules enabled
(https://github.com/fmtlib/fmt/pull/4347). Thanks @tkhyn.
- Fixed handling of named arguments in format specs
(https://github.com/fmtlib/fmt/issues/4360,
https://github.com/fmtlib/fmt/pull/4361). Thanks @dinomight.
- Added error reporting for duplicate named arguments
(https://github.com/fmtlib/fmt/pull/4367). Thanks @dinomight.
- Fixed formatting of `long` with `FMT_BUILTIN_TYPES=0`
(https://github.com/fmtlib/fmt/issues/4375,
https://github.com/fmtlib/fmt/issues/4394).
- Optimized `text_style` using bit packing
(https://github.com/fmtlib/fmt/pull/4363). Thanks @LocalSpook.
- Added support for incomplete types (https://github.com/fmtlib/fmt/issues/3180,
https://github.com/fmtlib/fmt/pull/4383). Thanks @LocalSpook.
- Fixed a flush issue in `fmt::print` when using libstdc++
(https://github.com/fmtlib/fmt/issues/4398).
- Fixed `fmt::println` usage with `FMT_ENFORCE_COMPILE_STRING` and legacy
compile-time checks (https://github.com/fmtlib/fmt/pull/4407).
Thanks @madmaxoft.
- Removed legacy header `fmt/core.h` from docs
(https://github.com/fmtlib/fmt/pull/4421,
https://github.com/fmtlib/fmt/pull/4422). Thanks @krzysztofkortas.
- Worked around limitations of `__builtin_strlen` during constant evaluation
(https://github.com/fmtlib/fmt/issues/4423,
https://github.com/fmtlib/fmt/pull/4429). Thanks @BRevzin.
- Worked around a bug in MSVC v141 (https://github.com/fmtlib/fmt/issues/4412,
https://github.com/fmtlib/fmt/pull/4413). Thanks @hirohira9119.
- Removed the `fmt_detail` namespace
(https://github.com/fmtlib/fmt/issues/4324).
- Removed specializations of `std::is_floating_point` in tests
(https://github.com/fmtlib/fmt/issues/4417).
- Fixed a CMake error when setting `CMAKE_MODULE_PATH` in the pedantic mode
(https://github.com/fmtlib/fmt/pull/4426). Thanks @rlalik.
- Updated the Bazel config (https://github.com/fmtlib/fmt/pull/4400).
Thanks @Vertexwahn.
# 11.1.4 - 2025-02-26
- Fixed ABI compatibility with earlier 11.x versions on Windows
(https://github.com/fmtlib/fmt/issues/4359).
- Improved the logic of switching between fixed and exponential format for
`float` (https://github.com/fmtlib/fmt/issues/3649).
- Moved `is_compiled_string` to the public API
(https://github.com/fmtlib/fmt/issues/4342). Thanks @SwooshyCueb.
- Simplified implementation of `operator""_cf`
(https://github.com/fmtlib/fmt/pull/4349). Thanks @LocalSpook.
- Fixed `__builtin_strlen` detection (https://github.com/fmtlib/fmt/pull/4329).
Thanks @LocalSpook.
- Fixed handling of BMI paths with the Ninja generator
(https://github.com/fmtlib/fmt/pull/4344). Thanks @tkhyn.
- Fixed gcc 8.3 compile errors (https://github.com/fmtlib/fmt/issues/4331,
https://github.com/fmtlib/fmt/pull/4336). Thanks @sergiud.
- Fixed a bogus MSVC warning (https://github.com/fmtlib/fmt/pull/4356).
Thanks @dinomight.
# 11.1.3 - 2025-01-25
- Fixed compilation on GCC 9.4 (https://github.com/fmtlib/fmt/issues/4313).
- Worked around an internal compiler error when using C++20 modules with GCC
14.2 and earlier (https://github.com/fmtlib/fmt/issues/4295).
- Worked around a bug in GCC 6 (https://github.com/fmtlib/fmt/issues/4318).
- Fixed an issue caused by instantiating `formatter<const T>`
(https://github.com/fmtlib/fmt/issues/4303,
https://github.com/fmtlib/fmt/pull/4325). Thanks @timsong-cpp.
- Fixed formatting into `std::ostreambuf_iterator` when using format string
compilation (https://github.com/fmtlib/fmt/issues/4309,
https://github.com/fmtlib/fmt/pull/4312). Thanks @phprus.
- Restored a constraint on the map formatter so that it correctly reports as
unformattable when the element is (https://github.com/fmtlib/fmt/pull/4326).
Thanks @timsong-cpp.
- Reduced the size of format specs (https://github.com/fmtlib/fmt/issues/4298).
- Readded `args()` to `fmt::format_context`
(https://github.com/fmtlib/fmt/issues/4307,
https://github.com/fmtlib/fmt/pull/4310). Thanks @Erroneous1.
- Fixed a bogus MSVC warning (https://github.com/fmtlib/fmt/issues/4314,
https://github.com/fmtlib/fmt/pull/4322). Thanks @ZehMatt.
- Fixed a pedantic mode error in the CMake config
(https://github.com/fmtlib/fmt/pull/4327). Thanks @rlalik.
# 11.1.2 - 2025-01-12
- Fixed ABI compatibility with earlier 11.x versions

View File

@@ -47,7 +47,7 @@ Try {fmt} in [Compiler Explorer](https://godbolt.org/z/8Mx1EW73v).
hundred million integers to strings per
second](http://www.zverovich.net/2020/06/13/fast-int-to-string-revisited.html)
- Small code size both in terms of source code with the minimum
configuration consisting of just three files, `core.h`, `format.h`
configuration consisting of just three files, `base.h`, `format.h`
and `format-inl.h`, and compiled code; see [Compile time and code
bloat](#compile-time-and-code-bloat)
- Reliability: the library has an extensive set of
@@ -74,7 +74,7 @@ See the [documentation](https://fmt.dev) for more details.
**Print to stdout** ([run](https://godbolt.org/z/Tevcjh))
``` c++
#include <fmt/core.h>
#include <fmt/base.h>
int main() {
fmt::print("Hello, world!\n");

View File

@@ -21,7 +21,7 @@
#endif
// The fmt library version in the form major * 10000 + minor * 100 + patch.
#define FMT_VERSION 110102
#define FMT_VERSION 110200
// Detect compiler versions.
#if defined(__clang__) && !defined(__ibmxl__)
@@ -96,9 +96,9 @@
// Detect C++14 relaxed constexpr.
#ifdef FMT_USE_CONSTEXPR
// Use the provided definition.
#elif FMT_GCC_VERSION >= 600 && FMT_CPLUSPLUS >= 201402L
// GCC only allows throw in constexpr since version 6:
// https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67371.
#elif FMT_GCC_VERSION >= 702 && FMT_CPLUSPLUS >= 201402L
// GCC only allows constexpr member functions in non-literal types since 7.2:
// https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66297.
# define FMT_USE_CONSTEXPR 1
#elif FMT_ICC_VERSION
# define FMT_USE_CONSTEXPR 0 // https://github.com/fmtlib/fmt/issues/1628
@@ -209,20 +209,6 @@
# define FMT_DEPRECATED /* deprecated */
#endif
#ifdef FMT_ALWAYS_INLINE
// Use the provided definition.
#elif FMT_GCC_VERSION || FMT_CLANG_VERSION
# define FMT_ALWAYS_INLINE inline __attribute__((always_inline))
#else
# define FMT_ALWAYS_INLINE inline
#endif
// A version of FMT_ALWAYS_INLINE to prevent code bloat in debug mode.
#ifdef NDEBUG
# define FMT_INLINE FMT_ALWAYS_INLINE
#else
# define FMT_INLINE inline
#endif
#if FMT_GCC_VERSION || FMT_CLANG_VERSION
# define FMT_VISIBILITY(value) __attribute__((visibility(value)))
#else
@@ -249,6 +235,28 @@
# define FMT_MSC_WARNING(...)
#endif
// Enable minimal optimizations for more compact code in debug mode.
FMT_PRAGMA_GCC(push_options)
#if !defined(__OPTIMIZE__) && !defined(__CUDACC__) && !defined(FMT_MODULE)
FMT_PRAGMA_GCC(optimize("Og"))
# define FMT_GCC_OPTIMIZED
#endif
FMT_PRAGMA_CLANG(diagnostic push)
#ifdef FMT_ALWAYS_INLINE
// Use the provided definition.
#elif FMT_GCC_VERSION || FMT_CLANG_VERSION
# define FMT_ALWAYS_INLINE inline __attribute__((always_inline))
#else
# define FMT_ALWAYS_INLINE inline
#endif
// A version of FMT_ALWAYS_INLINE to prevent code bloat in debug mode.
#if defined(NDEBUG) || defined(FMT_GCC_OPTIMIZED)
# define FMT_INLINE FMT_ALWAYS_INLINE
#else
# define FMT_INLINE inline
#endif
#ifndef FMT_BEGIN_NAMESPACE
# define FMT_BEGIN_NAMESPACE \
namespace fmt { \
@@ -294,15 +302,8 @@
#endif
#define FMT_APPLY_VARIADIC(expr) \
using ignore = int[]; \
(void)ignore { 0, (expr, 0)... }
// Enable minimal optimizations for more compact code in debug mode.
FMT_PRAGMA_GCC(push_options)
#if !defined(__OPTIMIZE__) && !defined(__CUDACC__)
FMT_PRAGMA_GCC(optimize("Og"))
#endif
FMT_PRAGMA_CLANG(diagnostic push)
using unused = int[]; \
(void)unused { 0, (expr, 0)... }
FMT_BEGIN_NAMESPACE
@@ -325,8 +326,8 @@ using underlying_t = typename std::underlying_type<T>::type;
template <typename T> using decay_t = typename std::decay<T>::type;
using nullptr_t = decltype(nullptr);
#if FMT_GCC_VERSION && FMT_GCC_VERSION < 500
// A workaround for gcc 4.9 to make void_t work in a SFINAE context.
#if (FMT_GCC_VERSION && FMT_GCC_VERSION < 500) || FMT_MSC_VERSION
// A workaround for gcc 4.9 & MSVC v141 to make void_t work in a SFINAE context.
template <typename...> struct void_t_impl {
using type = void;
};
@@ -526,20 +527,20 @@ template <typename Char> class basic_string_view {
constexpr basic_string_view() noexcept : data_(nullptr), size_(0) {}
/// Constructs a string reference object from a C string and a size.
/// Constructs a string view object from a C string and a size.
constexpr basic_string_view(const Char* s, size_t count) noexcept
: data_(s), size_(count) {}
constexpr basic_string_view(nullptr_t) = delete;
/// Constructs a string reference object from a C string.
/// Constructs a string view object from a C string.
#if FMT_GCC_VERSION
FMT_ALWAYS_INLINE
#endif
FMT_CONSTEXPR20 basic_string_view(const Char* s) : data_(s) {
#if FMT_HAS_BUILTIN(__buitin_strlen) || FMT_GCC_VERSION || FMT_CLANG_VERSION
if (std::is_same<Char, char>::value) {
size_ = __builtin_strlen(detail::narrow(s));
#if FMT_HAS_BUILTIN(__builtin_strlen) || FMT_GCC_VERSION || FMT_CLANG_VERSION
if (std::is_same<Char, char>::value && !detail::is_constant_evaluated()) {
size_ = __builtin_strlen(detail::narrow(s)); // strlen is not costexpr.
return;
}
#endif
@@ -548,7 +549,7 @@ template <typename Char> class basic_string_view {
size_ = len;
}
/// Constructs a string reference from a `std::basic_string` or a
/// Constructs a string view from a `std::basic_string` or a
/// `std::basic_string_view` object.
template <typename S,
FMT_ENABLE_IF(detail::is_std_string_like<S>::value&& std::is_same<
@@ -585,7 +586,6 @@ template <typename Char> class basic_string_view {
return starts_with(basic_string_view<Char>(s));
}
// Lexicographically compare this string reference to other.
FMT_CONSTEXPR auto compare(basic_string_view other) const -> int {
int result =
detail::compare(data_, other.data_, min_of(size_, other.size_));
@@ -616,7 +616,7 @@ template <typename Char> class basic_string_view {
using string_view = basic_string_view<char>;
/// Specifies if `T` is an extended character type. Can be specialized by users.
// DEPRECATED! Will be merged with is_char and moved to detail.
template <typename T> struct is_xchar : std::false_type {};
template <> struct is_xchar<wchar_t> : std::true_type {};
template <> struct is_xchar<char16_t> : std::true_type {};
@@ -625,7 +625,7 @@ template <> struct is_xchar<char32_t> : std::true_type {};
template <> struct is_xchar<char8_t> : std::true_type {};
#endif
// DEPRECATED! Will be replaced with an alias to prevent specializations.
// Specifies if `T` is a character (code unit) type.
template <typename T> struct is_char : is_xchar<T> {};
template <> struct is_char<char> : std::true_type {};
@@ -739,13 +739,15 @@ class basic_specs {
max_fill_size = 4
};
size_t data_ = 1 << fill_size_shift;
unsigned data_ = 1 << fill_size_shift;
static_assert(sizeof(basic_specs::data_) * CHAR_BIT >= 18, "");
// Character (code unit) type is erased to prevent template bloat.
char fill_data_[max_fill_size] = {' '};
FMT_CONSTEXPR void set_fill_size(size_t size) {
data_ = (data_ & ~fill_size_mask) | (size << fill_size_shift);
data_ = (data_ & ~fill_size_mask) |
(static_cast<unsigned>(size) << fill_size_shift);
}
public:
@@ -1030,6 +1032,11 @@ enum {
struct view {};
template <typename T, typename Enable = std::true_type>
struct is_view : std::false_type {};
template <typename T>
struct is_view<T, bool_constant<sizeof(T) != 0>> : std::is_base_of<view, T> {};
template <typename Char, typename T> struct named_arg;
template <typename T> struct is_named_arg : std::false_type {};
template <typename T> struct is_static_named_arg : std::false_type {};
@@ -1062,6 +1069,16 @@ template <typename Char> struct named_arg_info {
int id;
};
// named_args is non-const to suppress a bogus -Wmaybe-uninitalized in gcc 13.
template <typename Char>
FMT_CONSTEXPR void check_for_duplicate(named_arg_info<Char>* named_args,
int named_arg_index,
basic_string_view<Char> arg_name) {
for (int i = 0; i < named_arg_index; ++i) {
if (named_args[i].name == arg_name) report_error("duplicate named arg");
}
}
template <typename Char, typename T, FMT_ENABLE_IF(!is_named_arg<T>::value)>
void init_named_arg(named_arg_info<Char>*, int& arg_index, int&, const T&) {
++arg_index;
@@ -1069,6 +1086,7 @@ void init_named_arg(named_arg_info<Char>*, int& arg_index, int&, const T&) {
template <typename Char, typename T, FMT_ENABLE_IF(is_named_arg<T>::value)>
void init_named_arg(named_arg_info<Char>* named_args, int& arg_index,
int& named_arg_index, const T& arg) {
check_for_duplicate<Char>(named_args, named_arg_index, arg.name);
named_args[named_arg_index++] = {arg.name, arg_index++};
}
@@ -1082,12 +1100,13 @@ template <typename T, typename Char,
FMT_ENABLE_IF(is_static_named_arg<T>::value)>
FMT_CONSTEXPR void init_static_named_arg(named_arg_info<Char>* named_args,
int& arg_index, int& named_arg_index) {
check_for_duplicate<Char>(named_args, named_arg_index, T::name);
named_args[named_arg_index++] = {T::name, arg_index++};
}
// To minimize the number of types we need to deal with, long is translated
// either to int or to long long depending on its size.
enum { long_short = sizeof(long) == sizeof(int) };
enum { long_short = sizeof(long) == sizeof(int) && FMT_BUILTIN_TYPES };
using long_type = conditional_t<long_short, int, long long>;
using ulong_type = conditional_t<long_short, unsigned, unsigned long long>;
@@ -1119,7 +1138,7 @@ using use_formatter =
bool_constant<(std::is_class<T>::value || std::is_enum<T>::value ||
std::is_union<T>::value || std::is_array<T>::value) &&
!has_to_string_view<T>::value && !is_named_arg<T>::value &&
!use_format_as<T>::value && !use_format_as_member<T>::value>;
!use_format_as<T>::value && !use_format_as_member<U>::value>;
template <typename Char, typename T, typename U = remove_const_t<T>>
auto has_formatter_impl(T* p, buffered_context<Char>* ctx = nullptr)
@@ -1704,7 +1723,17 @@ class format_string_checker {
-> const Char* {
context_.advance_to(begin);
if (id >= 0 && id < NUM_ARGS) return parse_funcs_[id](context_);
while (begin != end && *begin != '}') ++begin;
// If id is out of range, it means we do not know the type and cannot parse
// the format at compile time. Instead, skip over content until we finish
// the format spec, accounting for any nested replacements.
for (int bracket_count = 0;
begin != end && (bracket_count > 0 || *begin != '}'); ++begin) {
if (*begin == '{')
++bracket_count;
else if (*begin == '}')
--bracket_count;
}
return begin;
}
@@ -2261,15 +2290,15 @@ template <> struct is_output_iterator<appender, char> : std::true_type {};
template <typename It, typename T>
struct is_output_iterator<
It, T,
void_t<decltype(*std::declval<decay_t<It>&>()++ = std::declval<T>())>>
: std::true_type {};
enable_if_t<std::is_assignable<decltype(*std::declval<decay_t<It>&>()++),
T>::value>> : std::true_type {};
#ifndef FMT_USE_LOCALE
# define FMT_USE_LOCALE (FMT_OPTIMIZE_SIZE <= 1)
#endif
// A type-erased reference to an std::locale to avoid a heavy <locale> include.
struct locale_ref {
class locale_ref {
#if FMT_USE_LOCALE
private:
const void* locale_; // A type-erased pointer to std::locale.
@@ -2281,6 +2310,7 @@ struct locale_ref {
inline explicit operator bool() const noexcept { return locale_ != nullptr; }
#endif // FMT_USE_LOCALE
public:
template <typename Locale> auto get() const -> Locale;
};
@@ -2654,6 +2684,7 @@ class context {
FMT_CONSTEXPR auto arg_id(string_view name) const -> int {
return args_.get_id(name);
}
auto args() const -> const format_args& { return args_; }
// Returns an iterator to the beginning of the output range.
FMT_CONSTEXPR auto out() const -> iterator { return out_; }
@@ -2699,7 +2730,7 @@ template <typename... T> struct fstring {
template <size_t N>
FMT_CONSTEVAL FMT_ALWAYS_INLINE fstring(const char (&s)[N]) : str(s, N - 1) {
using namespace detail;
static_assert(count<(std::is_base_of<view, remove_reference_t<T>>::value &&
static_assert(count<(is_view<remove_cvref_t<T>>::value &&
std::is_reference<T>::value)...>() == 0,
"passing views as lvalues is disallowed");
if (FMT_USE_CONSTEVAL) parse_format_string<char>(s, checker(s, arg_pack()));
@@ -2726,9 +2757,9 @@ template <typename... T> struct fstring {
std::is_same<typename S::char_type, char>::value)>
FMT_ALWAYS_INLINE fstring(const S&) : str(S()) {
FMT_CONSTEXPR auto sv = string_view(S());
FMT_CONSTEXPR int ignore =
FMT_CONSTEXPR int unused =
(parse_format_string(sv, checker(sv, arg_pack())), 0);
detail::ignore_unused(ignore);
detail::ignore_unused(unused);
}
fstring(runtime_format_string<> fmt) : str(fmt.str) {}

View File

@@ -22,21 +22,6 @@
#include "format.h"
namespace fmt_detail {
struct time_zone {
template <typename Duration, typename T>
auto to_sys(T)
-> std::chrono::time_point<std::chrono::system_clock, Duration> {
return {};
}
};
template <typename... T> inline auto current_zone(T...) -> time_zone* {
return nullptr;
}
template <typename... T> inline void _tzset(T...) {}
} // namespace fmt_detail
FMT_BEGIN_NAMESPACE
// Enable safe chrono durations, unless explicitly disabled.
@@ -435,14 +420,11 @@ auto write(OutputIt out, const std::tm& time, const std::locale& loc,
return write_encoded_tm_str(out, string_view(buf.data(), buf.size()), loc);
}
template <typename Rep1, typename Rep2>
struct is_same_arithmetic_type
: public std::integral_constant<bool,
(std::is_integral<Rep1>::value &&
std::is_integral<Rep2>::value) ||
(std::is_floating_point<Rep1>::value &&
std::is_floating_point<Rep2>::value)> {
};
template <typename T, typename U>
using is_similar_arithmetic_type =
bool_constant<(std::is_integral<T>::value && std::is_integral<U>::value) ||
(std::is_floating_point<T>::value &&
std::is_floating_point<U>::value)>;
FMT_NORETURN inline void throw_duration_error() {
FMT_THROW(format_error("cannot format duration"));
@@ -501,9 +483,9 @@ auto duration_cast(std::chrono::duration<FromRep, FromPeriod> from) -> To {
#endif
}
template <
typename To, typename FromRep, typename FromPeriod,
FMT_ENABLE_IF(!is_same_arithmetic_type<FromRep, typename To::rep>::value)>
template <typename To, typename FromRep, typename FromPeriod,
FMT_ENABLE_IF(
!is_similar_arithmetic_type<FromRep, typename To::rep>::value)>
auto duration_cast(std::chrono::duration<FromRep, FromPeriod> from) -> To {
// Mixed integer <-> float cast is not supported by safe_duration_cast.
return std::chrono::duration_cast<To>(from);
@@ -519,12 +501,30 @@ auto to_time_t(sys_time<Duration> time_point) -> std::time_t {
.count();
}
// Workaround a bug in libstdc++ which sets __cpp_lib_chrono to 201907 without
// providing current_zone(): https://github.com/fmtlib/fmt/issues/4160.
template <typename T> FMT_CONSTEXPR auto has_current_zone() -> bool {
using namespace std::chrono;
using namespace fmt_detail;
return !std::is_same<decltype(current_zone()), fmt_detail::time_zone*>::value;
namespace tz {
// DEPRECATED!
struct time_zone {
template <typename Duration, typename LocalTime>
auto to_sys(LocalTime) -> sys_time<Duration> {
return {};
}
};
template <typename... T> auto current_zone(T...) -> time_zone* {
return nullptr;
}
template <typename... T> void _tzset(T...) {}
} // namespace tz
// DEPRECATED!
inline void tzset_once() {
static bool init = []() {
using namespace tz;
_tzset();
return false;
}();
ignore_unused(init);
}
} // namespace detail
@@ -535,7 +535,7 @@ FMT_BEGIN_EXPORT
* expressed in local time. Unlike `std::localtime`, this function is
* thread-safe on most platforms.
*/
inline auto localtime(std::time_t time) -> std::tm {
FMT_DEPRECATED inline auto localtime(std::time_t time) -> std::tm {
struct dispatcher {
std::time_t time_;
std::tm tm_;
@@ -572,11 +572,11 @@ inline auto localtime(std::time_t time) -> std::tm {
}
#if FMT_USE_LOCAL_TIME
template <typename Duration,
FMT_ENABLE_IF(detail::has_current_zone<Duration>())>
inline auto localtime(std::chrono::local_time<Duration> time) -> std::tm {
template <typename Duration>
FMT_DEPRECATED auto localtime(std::chrono::local_time<Duration> time)
-> std::tm {
using namespace std::chrono;
using namespace fmt_detail;
using namespace detail::tz;
return localtime(detail::to_time_t(current_zone()->to_sys<Duration>(time)));
}
#endif
@@ -911,7 +911,14 @@ template <typename Derived> struct null_chrono_spec_handler {
FMT_CONSTEXPR void on_tz_name() { unsupported(); }
};
struct tm_format_checker : null_chrono_spec_handler<tm_format_checker> {
class tm_format_checker : public null_chrono_spec_handler<tm_format_checker> {
private:
bool has_timezone_ = false;
public:
constexpr explicit tm_format_checker(bool has_timezone)
: has_timezone_(has_timezone) {}
FMT_NORETURN inline void unsupported() {
FMT_THROW(format_error("no format"));
}
@@ -949,8 +956,12 @@ struct tm_format_checker : null_chrono_spec_handler<tm_format_checker> {
FMT_CONSTEXPR void on_24_hour_time() {}
FMT_CONSTEXPR void on_iso_time() {}
FMT_CONSTEXPR void on_am_pm() {}
FMT_CONSTEXPR void on_utc_offset(numeric_system) {}
FMT_CONSTEXPR void on_tz_name() {}
FMT_CONSTEXPR void on_utc_offset(numeric_system) {
if (!has_timezone_) FMT_THROW(format_error("no timezone"));
}
FMT_CONSTEXPR void on_tz_name() {
if (!has_timezone_) FMT_THROW(format_error("no timezone"));
}
};
inline auto tm_wday_full_name(int wday) -> const char* {
@@ -980,24 +991,27 @@ inline auto tm_mon_short_name(int mon) -> const char* {
}
template <typename T, typename = void>
struct has_member_data_tm_gmtoff : std::false_type {};
struct has_tm_gmtoff : std::false_type {};
template <typename T>
struct has_member_data_tm_gmtoff<T, void_t<decltype(T::tm_gmtoff)>>
: std::true_type {};
struct has_tm_gmtoff<T, void_t<decltype(T::tm_gmtoff)>> : std::true_type {};
template <typename T, typename = void>
struct has_member_data_tm_zone : std::false_type {};
template <typename T, typename = void> struct has_tm_zone : std::false_type {};
template <typename T>
struct has_member_data_tm_zone<T, void_t<decltype(T::tm_zone)>>
: std::true_type {};
struct has_tm_zone<T, void_t<decltype(T::tm_zone)>> : std::true_type {};
inline void tzset_once() {
static bool init = []() {
using namespace fmt_detail;
_tzset();
return false;
}();
ignore_unused(init);
template <typename T, FMT_ENABLE_IF(has_tm_zone<T>::value)>
bool set_tm_zone(T& time, char* tz) {
time.tm_zone = tz;
return true;
}
template <typename T, FMT_ENABLE_IF(!has_tm_zone<T>::value)>
bool set_tm_zone(T&, char*) {
return false;
}
inline char* utc() {
static char tz[] = "UTC";
return tz;
}
// Converts value to Int and checks that it's in the range [0, upper).
@@ -1005,7 +1019,7 @@ template <typename T, typename Int, FMT_ENABLE_IF(std::is_integral<T>::value)>
inline auto to_nonnegative_int(T value, Int upper) -> Int {
if (!std::is_unsigned<Int>::value &&
(value < 0 || to_unsigned(value) > to_unsigned(upper))) {
FMT_THROW(fmt::format_error("chrono value is out of range"));
FMT_THROW(format_error("chrono value is out of range"));
}
return static_cast<Int>(value);
}
@@ -1090,7 +1104,7 @@ void write_fractional_seconds(OutputIt& out, Duration d, int precision = -1) {
// Format subseconds which are given as a floating point type with an
// appropriate number of digits. We cannot pass the Duration here, as we
// explicitly need to pass the Rep value in the chrono_formatter.
// explicitly need to pass the Rep value in the duration_formatter.
template <typename Duration>
void write_floating_seconds(memory_buffer& buf, Duration duration,
int num_fractional_digits = -1) {
@@ -1124,7 +1138,7 @@ class tm_writer {
static constexpr int days_per_week = 7;
const std::locale& loc_;
const bool is_classic_;
bool is_classic_;
OutputIt out_;
const Duration* subsecs_;
const std::tm& tm_;
@@ -1160,8 +1174,8 @@ class tm_writer {
}
auto tm_hour12() const noexcept -> int {
const auto h = tm_hour();
const auto z = h < 12 ? h : h - 12;
auto h = tm_hour();
auto z = h < 12 ? h : h - 12;
return z == 0 ? 12 : z;
}
@@ -1177,11 +1191,11 @@ class tm_writer {
// Algorithm: https://en.wikipedia.org/wiki/ISO_week_date.
auto iso_year_weeks(long long curr_year) const noexcept -> int {
const auto prev_year = curr_year - 1;
const auto curr_p =
auto prev_year = curr_year - 1;
auto curr_p =
(curr_year + curr_year / 4 - curr_year / 100 + curr_year / 400) %
days_per_week;
const auto prev_p =
auto prev_p =
(prev_year + prev_year / 4 - prev_year / 100 + prev_year / 400) %
days_per_week;
return 52 + ((curr_p == 4 || prev_p == 3) ? 1 : 0);
@@ -1191,15 +1205,15 @@ class tm_writer {
days_per_week;
}
auto tm_iso_week_year() const noexcept -> long long {
const auto year = tm_year();
const auto w = iso_week_num(tm_yday(), tm_wday());
auto year = tm_year();
auto w = iso_week_num(tm_yday(), tm_wday());
if (w < 1) return year - 1;
if (w > iso_year_weeks(year)) return year + 1;
return year;
}
auto tm_iso_week_of_year() const noexcept -> int {
const auto year = tm_year();
const auto w = iso_week_num(tm_yday(), tm_wday());
auto year = tm_year();
auto w = iso_week_num(tm_yday(), tm_wday());
if (w < 1) return iso_year_weeks(year - 1);
if (w > iso_year_weeks(year)) return 1;
return w;
@@ -1236,9 +1250,8 @@ class tm_writer {
uint32_or_64_or_128_t<long long> n = to_unsigned(year);
const int num_digits = count_digits(n);
if (negative && pad == pad_type::zero) *out_++ = '-';
if (width > num_digits) {
if (width > num_digits)
out_ = detail::write_padding(out_, pad, width - num_digits);
}
if (negative && pad != pad_type::zero) *out_++ = '-';
out_ = format_decimal<Char>(out_, n, num_digits);
}
@@ -1259,45 +1272,22 @@ class tm_writer {
write2(static_cast<int>(offset % 60));
}
template <typename T, FMT_ENABLE_IF(has_member_data_tm_gmtoff<T>::value)>
void format_utc_offset_impl(const T& tm, numeric_system ns) {
template <typename T, FMT_ENABLE_IF(has_tm_gmtoff<T>::value)>
void format_utc_offset(const T& tm, numeric_system ns) {
write_utc_offset(tm.tm_gmtoff, ns);
}
template <typename T, FMT_ENABLE_IF(!has_member_data_tm_gmtoff<T>::value)>
void format_utc_offset_impl(const T& tm, numeric_system ns) {
#if defined(_WIN32) && defined(_UCRT)
tzset_once();
long offset = 0;
_get_timezone(&offset);
if (tm.tm_isdst) {
long dstbias = 0;
_get_dstbias(&dstbias);
offset += dstbias;
}
write_utc_offset(-offset, ns);
#else
if (ns == numeric_system::standard) return format_localized('z');
// Extract timezone offset from timezone conversion functions.
std::tm gtm = tm;
std::time_t gt = std::mktime(&gtm);
std::tm ltm = gmtime(gt);
std::time_t lt = std::mktime(&ltm);
long long offset = gt - lt;
write_utc_offset(offset, ns);
#endif
template <typename T, FMT_ENABLE_IF(!has_tm_gmtoff<T>::value)>
void format_utc_offset(const T&, numeric_system ns) {
write_utc_offset(0, ns);
}
template <typename T, FMT_ENABLE_IF(has_member_data_tm_zone<T>::value)>
void format_tz_name_impl(const T& tm) {
if (is_classic_)
out_ = write_tm_str<Char>(out_, tm.tm_zone, loc_);
else
format_localized('Z');
template <typename T, FMT_ENABLE_IF(has_tm_zone<T>::value)>
void format_tz_name(const T& tm) {
out_ = write_tm_str<Char>(out_, tm.tm_zone, loc_);
}
template <typename T, FMT_ENABLE_IF(!has_member_data_tm_zone<T>::value)>
void format_tz_name_impl(const T&) {
format_localized('Z');
template <typename T, FMT_ENABLE_IF(!has_tm_zone<T>::value)>
void format_tz_name(const T&) {
out_ = std::copy_n(utc(), 3, out_);
}
void format_localized(char format, char modifier = 0) {
@@ -1408,8 +1398,8 @@ class tm_writer {
out_ = copy<Char>(std::begin(buf) + offset, std::end(buf), out_);
}
void on_utc_offset(numeric_system ns) { format_utc_offset_impl(tm_, ns); }
void on_tz_name() { format_tz_name_impl(tm_); }
void on_utc_offset(numeric_system ns) { format_utc_offset(tm_, ns); }
void on_tz_name() { format_tz_name(tm_); }
void on_year(numeric_system ns, pad_type pad) {
if (is_classic_ || ns == numeric_system::standard)
@@ -1483,11 +1473,10 @@ class tm_writer {
void on_day_of_year(pad_type pad) {
auto yday = tm_yday() + 1;
auto digit1 = yday / 100;
if (digit1 != 0) {
if (digit1 != 0)
write1(digit1);
} else {
else
out_ = detail::write_padding(out_, pad);
}
write2(yday % 100, pad);
}
@@ -1624,18 +1613,16 @@ template <typename Rep, typename Period,
FMT_ENABLE_IF(std::is_integral<Rep>::value)>
inline auto get_milliseconds(std::chrono::duration<Rep, Period> d)
-> std::chrono::duration<Rep, std::milli> {
// this may overflow and/or the result may not fit in the
// target type.
// This may overflow and/or the result may not fit in the target type.
#if FMT_SAFE_DURATION_CAST
using CommonSecondsType =
using common_seconds_type =
typename std::common_type<decltype(d), std::chrono::seconds>::type;
const auto d_as_common = detail::duration_cast<CommonSecondsType>(d);
const auto d_as_whole_seconds =
auto d_as_common = detail::duration_cast<common_seconds_type>(d);
auto d_as_whole_seconds =
detail::duration_cast<std::chrono::seconds>(d_as_common);
// this conversion should be nonproblematic
const auto diff = d_as_common - d_as_whole_seconds;
const auto ms =
detail::duration_cast<std::chrono::duration<Rep, std::milli>>(diff);
// This conversion should be nonproblematic.
auto diff = d_as_common - d_as_whole_seconds;
auto ms = detail::duration_cast<std::chrono::duration<Rep, std::milli>>(diff);
return ms;
#else
auto s = detail::duration_cast<std::chrono::seconds>(d);
@@ -1707,32 +1694,28 @@ class get_locale {
}
};
template <typename FormatContext, typename OutputIt, typename Rep,
typename Period>
struct chrono_formatter {
FormatContext& context;
OutputIt out;
int precision;
bool localized = false;
template <typename Char, typename Rep, typename Period>
struct duration_formatter {
using iterator = basic_appender<Char>;
iterator out;
// rep is unsigned to avoid overflow.
using rep =
conditional_t<std::is_integral<Rep>::value && sizeof(Rep) < sizeof(int),
unsigned, typename make_unsigned_or_unchanged<Rep>::type>;
rep val;
int precision;
locale_ref locale;
bool localized = false;
using seconds = std::chrono::duration<rep>;
seconds s;
using milliseconds = std::chrono::duration<rep, std::milli>;
bool negative;
using char_type = typename FormatContext::char_type;
using tm_writer_type = tm_writer<OutputIt, char_type>;
using tm_writer_type = tm_writer<iterator, Char>;
chrono_formatter(FormatContext& ctx, OutputIt o,
std::chrono::duration<Rep, Period> d)
: context(ctx),
out(o),
val(static_cast<rep>(d.count())),
negative(false) {
duration_formatter(iterator o, std::chrono::duration<Rep, Period> d,
locale_ref loc)
: out(o), val(static_cast<rep>(d.count())), locale(loc), negative(false) {
if (d.count() < 0) {
val = 0 - val;
negative = true;
@@ -1746,19 +1729,16 @@ struct chrono_formatter {
// returns true if nan or inf, writes to out.
auto handle_nan_inf() -> bool {
if (isfinite(val)) {
return false;
}
if (isfinite(val)) return false;
if (isnan(val)) {
write_nan();
return true;
}
// must be +-inf
if (val > 0) {
write_pinf();
} else {
write_ninf();
}
if (val > 0)
std::copy_n("inf", 3, out);
else
std::copy_n("-inf", 4, out);
return true;
}
@@ -1786,10 +1766,9 @@ struct chrono_formatter {
}
void write_sign() {
if (negative) {
*out++ = '-';
negative = false;
}
if (!negative) return;
*out++ = '-';
negative = false;
}
void write(Rep value, int width, pad_type pad = pad_type::zero) {
@@ -1801,24 +1780,22 @@ struct chrono_formatter {
if (width > num_digits) {
out = detail::write_padding(out, pad, width - num_digits);
}
out = format_decimal<char_type>(out, n, num_digits);
out = format_decimal<Char>(out, n, num_digits);
}
void write_nan() { std::copy_n("nan", 3, out); }
void write_pinf() { std::copy_n("inf", 3, out); }
void write_ninf() { std::copy_n("-inf", 4, out); }
template <typename Callback, typename... Args>
void format_tm(const tm& time, Callback cb, Args... args) {
if (isnan(val)) return write_nan();
get_locale loc(localized, context.locale());
get_locale loc(localized, locale);
auto w = tm_writer_type(loc, out, time);
(w.*cb)(args...);
out = w.out();
}
void on_text(const char_type* begin, const char_type* end) {
copy<char_type>(begin, end, out);
void on_text(const Char* begin, const Char* end) {
copy<Char>(begin, end, out);
}
// These are not implemented because durations don't have date information.
@@ -1888,13 +1865,12 @@ struct chrono_formatter {
write_floating_seconds(buf, std::chrono::duration<rep, Period>(val),
precision);
if (negative) *out++ = '-';
if (buf.size() < 2 || buf[1] == '.') {
if (buf.size() < 2 || buf[1] == '.')
out = detail::write_padding(out, pad);
}
out = copy<char_type>(buf.begin(), buf.end(), out);
out = copy<Char>(buf.begin(), buf.end(), out);
} else {
write(second(), 2, pad);
write_fractional_seconds<char_type>(
write_fractional_seconds<Char>(
out, std::chrono::duration<rep, Period>(val), precision);
}
return;
@@ -1936,12 +1912,10 @@ struct chrono_formatter {
void on_duration_value() {
if (handle_nan_inf()) return;
write_sign();
out = format_duration_value<char_type>(out, val, precision);
out = format_duration_value<Char>(out, val, precision);
}
void on_duration_unit() {
out = format_duration_unit<char_type, Period>(out);
}
void on_duration_unit() { out = format_duration_unit<Char, Period>(out); }
};
} // namespace detail
@@ -2011,12 +1985,11 @@ class year_month_day {
constexpr auto month() const noexcept -> fmt::month { return month_; }
constexpr auto day() const noexcept -> fmt::day { return day_; }
};
#endif
#endif // __cpp_lib_chrono >= 201907
template <typename Char>
struct formatter<weekday, Char> : private formatter<std::tm, Char> {
private:
bool localized_ = false;
bool use_tm_formatter_ = false;
public:
@@ -2024,8 +1997,7 @@ struct formatter<weekday, Char> : private formatter<std::tm, Char> {
auto it = ctx.begin(), end = ctx.end();
if (it != end && *it == 'L') {
++it;
localized_ = true;
return it;
this->set_localized();
}
use_tm_formatter_ = it != end && *it != '}';
return use_tm_formatter_ ? formatter<std::tm, Char>::parse(ctx) : it;
@@ -2036,7 +2008,7 @@ struct formatter<weekday, Char> : private formatter<std::tm, Char> {
auto time = std::tm();
time.tm_wday = static_cast<int>(wd.c_encoding());
if (use_tm_formatter_) return formatter<std::tm, Char>::format(time, ctx);
detail::get_locale loc(localized_, ctx.locale());
detail::get_locale loc(this->localized(), ctx.locale());
auto w = detail::tm_writer<decltype(ctx.out()), Char>(loc, ctx.out(), time);
w.on_abbr_weekday();
return w.out();
@@ -2070,7 +2042,6 @@ struct formatter<day, Char> : private formatter<std::tm, Char> {
template <typename Char>
struct formatter<month, Char> : private formatter<std::tm, Char> {
private:
bool localized_ = false;
bool use_tm_formatter_ = false;
public:
@@ -2078,8 +2049,7 @@ struct formatter<month, Char> : private formatter<std::tm, Char> {
auto it = ctx.begin(), end = ctx.end();
if (it != end && *it == 'L') {
++it;
localized_ = true;
return it;
this->set_localized();
}
use_tm_formatter_ = it != end && *it != '}';
return use_tm_formatter_ ? formatter<std::tm, Char>::parse(ctx) : it;
@@ -2090,7 +2060,7 @@ struct formatter<month, Char> : private formatter<std::tm, Char> {
auto time = std::tm();
time.tm_mon = static_cast<int>(static_cast<unsigned>(m)) - 1;
if (use_tm_formatter_) return formatter<std::tm, Char>::format(time, ctx);
detail::get_locale loc(localized_, ctx.locale());
detail::get_locale loc(this->localized(), ctx.locale());
auto w = detail::tm_writer<decltype(ctx.out()), Char>(loc, ctx.out(), time);
w.on_abbr_month();
return w.out();
@@ -2154,7 +2124,6 @@ struct formatter<std::chrono::duration<Rep, Period>, Char> {
format_specs specs_;
detail::arg_ref<Char> width_ref_;
detail::arg_ref<Char> precision_ref_;
bool localized_ = false;
basic_string_view<Char> fmt_;
public:
@@ -2177,7 +2146,7 @@ struct formatter<std::chrono::duration<Rep, Period>, Char> {
it = detail::parse_precision(it, end, specs_, precision_ref_, ctx);
}
if (it != end && *it == 'L') {
localized_ = true;
specs_.set_localized();
++it;
}
end = detail::parse_chrono_format(it, end, checker);
@@ -2204,11 +2173,10 @@ struct formatter<std::chrono::duration<Rep, Period>, Char> {
out = detail::format_duration_value<Char>(out, d.count(), precision);
detail::format_duration_unit<Char, Period>(out);
} else {
using chrono_formatter =
detail::chrono_formatter<FormatContext, decltype(out), Rep, Period>;
auto f = chrono_formatter(ctx, out, d);
auto f =
detail::duration_formatter<Char, Rep, Period>(out, d, ctx.locale());
f.precision = precision;
f.localized = localized_;
f.localized = specs_.localized();
detail::parse_chrono_format(begin, end, f);
}
return detail::write(
@@ -2220,30 +2188,15 @@ template <typename Char> struct formatter<std::tm, Char> {
private:
format_specs specs_;
detail::arg_ref<Char> width_ref_;
basic_string_view<Char> fmt_ =
detail::string_literal<Char, '%', 'F', ' ', '%', 'T'>();
protected:
basic_string_view<Char> fmt_;
auto localized() const -> bool { return specs_.localized(); }
FMT_CONSTEXPR void set_localized() { specs_.set_localized(); }
template <typename Duration, typename FormatContext>
auto do_format(const std::tm& tm, FormatContext& ctx,
const Duration* subsecs) const -> decltype(ctx.out()) {
auto specs = specs_;
auto buf = basic_memory_buffer<Char>();
auto out = basic_appender<Char>(buf);
detail::handle_dynamic_spec(specs.dynamic_width(), specs.width, width_ref_,
ctx);
auto loc_ref = ctx.locale();
detail::get_locale loc(static_cast<bool>(loc_ref), loc_ref);
auto w =
detail::tm_writer<decltype(out), Char, Duration>(loc, out, tm, subsecs);
detail::parse_chrono_format(fmt_.begin(), fmt_.end(), w);
return detail::write(
ctx.out(), basic_string_view<Char>(buf.data(), buf.size()), specs);
}
public:
FMT_CONSTEXPR auto parse(parse_context<Char>& ctx) -> const Char* {
FMT_CONSTEXPR auto do_parse(parse_context<Char>& ctx, bool has_timezone)
-> const Char* {
auto it = ctx.begin(), end = ctx.end();
if (it == end || *it == '}') return it;
@@ -2256,12 +2209,41 @@ template <typename Char> struct formatter<std::tm, Char> {
if (it == end) return it;
}
end = detail::parse_chrono_format(it, end, detail::tm_format_checker());
if (*it == 'L') {
specs_.set_localized();
++it;
}
end = detail::parse_chrono_format(it, end,
detail::tm_format_checker(has_timezone));
// Replace the default format string only if the new spec is not empty.
if (end != it) fmt_ = {it, detail::to_unsigned(end - it)};
return end;
}
template <typename Duration, typename FormatContext>
auto do_format(const std::tm& tm, FormatContext& ctx,
const Duration* subsecs) const -> decltype(ctx.out()) {
auto specs = specs_;
auto buf = basic_memory_buffer<Char>();
auto out = basic_appender<Char>(buf);
detail::handle_dynamic_spec(specs.dynamic_width(), specs.width, width_ref_,
ctx);
auto loc_ref = specs.localized() ? ctx.locale() : detail::locale_ref();
detail::get_locale loc(static_cast<bool>(loc_ref), loc_ref);
auto w = detail::tm_writer<basic_appender<Char>, Char, Duration>(
loc, out, tm, subsecs);
detail::parse_chrono_format(fmt_.begin(), fmt_.end(), w);
return detail::write(
ctx.out(), basic_string_view<Char>(buf.data(), buf.size()), specs);
}
public:
FMT_CONSTEXPR auto parse(parse_context<Char>& ctx) -> const Char* {
return do_parse(ctx, detail::has_tm_gmtoff<std::tm>::value);
}
template <typename FormatContext>
auto format(const std::tm& tm, FormatContext& ctx) const
-> decltype(ctx.out()) {
@@ -2269,10 +2251,11 @@ template <typename Char> struct formatter<std::tm, Char> {
}
};
// DEPRECATED! Reversed order of template parameters.
template <typename Char, typename Duration>
struct formatter<sys_time<Duration>, Char> : formatter<std::tm, Char> {
FMT_CONSTEXPR formatter() {
this->fmt_ = detail::string_literal<Char, '%', 'F', ' ', '%', 'T'>();
struct formatter<sys_time<Duration>, Char> : private formatter<std::tm, Char> {
FMT_CONSTEXPR auto parse(parse_context<Char>& ctx) -> const Char* {
return this->do_parse(ctx, true);
}
template <typename FormatContext>
@@ -2283,6 +2266,7 @@ struct formatter<sys_time<Duration>, Char> : formatter<std::tm, Char> {
if (detail::const_check(
period::num == 1 && period::den == 1 &&
!std::is_floating_point<typename Duration::rep>::value)) {
detail::set_tm_zone(tm, detail::utc());
return formatter<std::tm, Char>::format(tm, ctx);
}
Duration epoch = val.time_since_epoch();
@@ -2290,11 +2274,13 @@ struct formatter<sys_time<Duration>, Char> : formatter<std::tm, Char> {
epoch - detail::duration_cast<std::chrono::seconds>(epoch));
if (subsecs.count() < 0) {
auto second = detail::duration_cast<Duration>(std::chrono::seconds(1));
if (tm.tm_sec != 0)
if (tm.tm_sec != 0) {
--tm.tm_sec;
else
} else {
tm = gmtime(val - second);
subsecs += detail::duration_cast<Duration>(std::chrono::seconds(1));
detail::set_tm_zone(tm, detail::utc());
}
subsecs += second;
}
return formatter<std::tm, Char>::do_format(tm, ctx, &subsecs);
}
@@ -2312,23 +2298,29 @@ struct formatter<utc_time<Duration>, Char>
};
template <typename Duration, typename Char>
struct formatter<local_time<Duration>, Char> : formatter<std::tm, Char> {
FMT_CONSTEXPR formatter() {
this->fmt_ = detail::string_literal<Char, '%', 'F', ' ', '%', 'T'>();
struct formatter<local_time<Duration>, Char>
: private formatter<std::tm, Char> {
FMT_CONSTEXPR auto parse(parse_context<Char>& ctx) -> const Char* {
return this->do_parse(ctx, false);
}
template <typename FormatContext>
auto format(local_time<Duration> val, FormatContext& ctx) const
-> decltype(ctx.out()) {
auto time_since_epoch = val.time_since_epoch();
auto seconds_since_epoch =
detail::duration_cast<std::chrono::seconds>(time_since_epoch);
// Use gmtime to prevent time zone conversion since local_time has an
// unspecified time zone.
std::tm t = gmtime(seconds_since_epoch.count());
using period = typename Duration::period;
if (period::num == 1 && period::den == 1 &&
!std::is_floating_point<typename Duration::rep>::value) {
return formatter<std::tm, Char>::format(localtime(val), ctx);
return formatter<std::tm, Char>::format(t, ctx);
}
auto epoch = val.time_since_epoch();
auto subsecs = detail::duration_cast<Duration>(
epoch - detail::duration_cast<std::chrono::seconds>(epoch));
return formatter<std::tm, Char>::do_format(localtime(val), ctx, &subsecs);
auto subsecs =
detail::duration_cast<Duration>(time_since_epoch - seconds_since_epoch);
return formatter<std::tm, Char>::do_format(t, ctx, &subsecs);
}
};

View File

@@ -190,11 +190,11 @@ enum class emphasis : uint8_t {
// rgb is a struct for red, green and blue colors.
// Using the name "rgb" makes some editors show the color in a tooltip.
struct rgb {
FMT_CONSTEXPR rgb() : r(0), g(0), b(0) {}
FMT_CONSTEXPR rgb(uint8_t r_, uint8_t g_, uint8_t b_) : r(r_), g(g_), b(b_) {}
FMT_CONSTEXPR rgb(uint32_t hex)
constexpr rgb() : r(0), g(0), b(0) {}
constexpr rgb(uint8_t r_, uint8_t g_, uint8_t b_) : r(r_), g(g_), b(b_) {}
constexpr rgb(uint32_t hex)
: r((hex >> 16) & 0xFF), g((hex >> 8) & 0xFF), b(hex & 0xFF) {}
FMT_CONSTEXPR rgb(color hex)
constexpr rgb(color hex)
: r((uint32_t(hex) >> 16) & 0xFF),
g((uint32_t(hex) >> 8) & 0xFF),
b(uint32_t(hex) & 0xFF) {}
@@ -205,97 +205,135 @@ struct rgb {
namespace detail {
// color is a struct of either a rgb color or a terminal color.
// A bit-packed variant of an RGB color, a terminal color, or unset color.
// see text_style for the bit-packing scheme.
struct color_type {
FMT_CONSTEXPR color_type() noexcept : is_rgb(), value{} {}
FMT_CONSTEXPR color_type(color rgb_color) noexcept : is_rgb(true), value{} {
value.rgb_color = static_cast<uint32_t>(rgb_color);
constexpr color_type() noexcept = default;
constexpr color_type(color rgb_color) noexcept
: value_(static_cast<uint32_t>(rgb_color) | (1 << 24)) {}
constexpr color_type(rgb rgb_color) noexcept
: color_type(static_cast<color>(
(static_cast<uint32_t>(rgb_color.r) << 16) |
(static_cast<uint32_t>(rgb_color.g) << 8) | rgb_color.b)) {}
constexpr color_type(terminal_color term_color) noexcept
: value_(static_cast<uint32_t>(term_color) | (3 << 24)) {}
constexpr auto is_terminal_color() const noexcept -> bool {
return (value_ & (1 << 25)) != 0;
}
FMT_CONSTEXPR color_type(rgb rgb_color) noexcept : is_rgb(true), value{} {
value.rgb_color = (static_cast<uint32_t>(rgb_color.r) << 16) |
(static_cast<uint32_t>(rgb_color.g) << 8) | rgb_color.b;
constexpr auto value() const noexcept -> uint32_t {
return value_ & 0xFFFFFF;
}
FMT_CONSTEXPR color_type(terminal_color term_color) noexcept
: is_rgb(), value{} {
value.term_color = static_cast<uint8_t>(term_color);
}
bool is_rgb;
union color_union {
uint8_t term_color;
uint32_t rgb_color;
} value;
constexpr color_type(uint32_t value) noexcept : value_(value) {}
uint32_t value_ = 0;
};
} // namespace detail
/// A text style consisting of foreground and background colors and emphasis.
class text_style {
// The information is packed as follows:
// ┌──┐
// │ 0│─┐
// │..│ ├── foreground color value
// │23│─┘
// ├──┤
// │24│─┬── discriminator for the above value. 00 if unset, 01 if it's
// │25│─┘ an RGB color, or 11 if it's a terminal color (10 is unused)
// ├──┤
// │26│──── overflow bit, always zero (see below)
// ├──┤
// │27│─┐
// │..│ │
// │50│ │
// ├──┤ │
// │51│ ├── background color (same format as the foreground color)
// │52│ │
// ├──┤ │
// │53│─┘
// ├──┤
// │54│─┐
// │..│ ├── emphases
// │61│─┘
// ├──┤
// │62│─┬── unused
// │63│─┘
// └──┘
// The overflow bits are there to make operator|= efficient.
// When ORing, we must throw if, for either the foreground or background,
// one style specifies a terminal color and the other specifies any color
// (terminal or RGB); in other words, if one discriminator is 11 and the
// other is 11 or 01.
//
// We do that check by adding the styles. Consider what adding does to each
// possible pair of discriminators:
// 00 + 00 = 000
// 01 + 00 = 001
// 11 + 00 = 011
// 01 + 01 = 010
// 11 + 01 = 100 (!!)
// 11 + 11 = 110 (!!)
// In the last two cases, the ones we want to catch, the third bit——the
// overflow bit——is set. Bingo.
//
// We must take into account the possible carry bit from the bits
// before the discriminator. The only potentially problematic case is
// 11 + 00 = 011 (a carry bit would make it 100, not good!), but a carry
// bit is impossible in that case, because 00 (unset color) means the
// 24 bits that precede the discriminator are all zero.
//
// This test can be applied to both colors simultaneously.
public:
FMT_CONSTEXPR text_style(emphasis em = emphasis()) noexcept
: set_foreground_color(), set_background_color(), ems(em) {}
: style_(static_cast<uint64_t>(em) << 54) {}
FMT_CONSTEXPR auto operator|=(const text_style& rhs) -> text_style& {
if (!set_foreground_color) {
set_foreground_color = rhs.set_foreground_color;
foreground_color = rhs.foreground_color;
} else if (rhs.set_foreground_color) {
if (!foreground_color.is_rgb || !rhs.foreground_color.is_rgb)
report_error("can't OR a terminal color");
foreground_color.value.rgb_color |= rhs.foreground_color.value.rgb_color;
}
if (!set_background_color) {
set_background_color = rhs.set_background_color;
background_color = rhs.background_color;
} else if (rhs.set_background_color) {
if (!background_color.is_rgb || !rhs.background_color.is_rgb)
report_error("can't OR a terminal color");
background_color.value.rgb_color |= rhs.background_color.value.rgb_color;
}
ems = static_cast<emphasis>(static_cast<uint8_t>(ems) |
static_cast<uint8_t>(rhs.ems));
FMT_CONSTEXPR auto operator|=(text_style rhs) -> text_style& {
if (((style_ + rhs.style_) & ((1ULL << 26) | (1ULL << 53))) != 0)
report_error("can't OR a terminal color");
style_ |= rhs.style_;
return *this;
}
friend FMT_CONSTEXPR auto operator|(text_style lhs, const text_style& rhs)
friend FMT_CONSTEXPR auto operator|(text_style lhs, text_style rhs)
-> text_style {
return lhs |= rhs;
}
FMT_CONSTEXPR auto operator==(text_style rhs) const noexcept -> bool {
return style_ == rhs.style_;
}
FMT_CONSTEXPR auto operator!=(text_style rhs) const noexcept -> bool {
return !(*this == rhs);
}
FMT_CONSTEXPR auto has_foreground() const noexcept -> bool {
return set_foreground_color;
return (style_ & (1 << 24)) != 0;
}
FMT_CONSTEXPR auto has_background() const noexcept -> bool {
return set_background_color;
return (style_ & (1ULL << 51)) != 0;
}
FMT_CONSTEXPR auto has_emphasis() const noexcept -> bool {
return static_cast<uint8_t>(ems) != 0;
return (style_ >> 54) != 0;
}
FMT_CONSTEXPR auto get_foreground() const noexcept -> detail::color_type {
FMT_ASSERT(has_foreground(), "no foreground specified for this style");
return foreground_color;
return style_ & 0x3FFFFFF;
}
FMT_CONSTEXPR auto get_background() const noexcept -> detail::color_type {
FMT_ASSERT(has_background(), "no background specified for this style");
return background_color;
return (style_ >> 27) & 0x3FFFFFF;
}
FMT_CONSTEXPR auto get_emphasis() const noexcept -> emphasis {
FMT_ASSERT(has_emphasis(), "no emphasis specified for this style");
return ems;
return static_cast<emphasis>(style_ >> 54);
}
private:
FMT_CONSTEXPR text_style(bool is_foreground,
detail::color_type text_color) noexcept
: set_foreground_color(), set_background_color(), ems() {
if (is_foreground) {
foreground_color = text_color;
set_foreground_color = true;
} else {
background_color = text_color;
set_background_color = true;
}
}
FMT_CONSTEXPR text_style(uint64_t style) noexcept : style_(style) {}
friend FMT_CONSTEXPR auto fg(detail::color_type foreground) noexcept
-> text_style;
@@ -303,23 +341,19 @@ class text_style {
friend FMT_CONSTEXPR auto bg(detail::color_type background) noexcept
-> text_style;
detail::color_type foreground_color;
detail::color_type background_color;
bool set_foreground_color;
bool set_background_color;
emphasis ems;
uint64_t style_ = 0;
};
/// Creates a text style from the foreground (text) color.
FMT_CONSTEXPR inline auto fg(detail::color_type foreground) noexcept
-> text_style {
return text_style(true, foreground);
return foreground.value_;
}
/// Creates a text style from the background color.
FMT_CONSTEXPR inline auto bg(detail::color_type background) noexcept
-> text_style {
return text_style(false, background);
return static_cast<uint64_t>(background.value_) << 27;
}
FMT_CONSTEXPR inline auto operator|(emphasis lhs, emphasis rhs) noexcept
@@ -334,9 +368,9 @@ template <typename Char> struct ansi_color_escape {
const char* esc) noexcept {
// If we have a terminal color, we need to output another escape code
// sequence.
if (!text_color.is_rgb) {
if (text_color.is_terminal_color()) {
bool is_background = esc == string_view("\x1b[48;2;");
uint32_t value = text_color.value.term_color;
uint32_t value = text_color.value();
// Background ASCII codes are the same as the foreground ones but with
// 10 more.
if (is_background) value += 10u;
@@ -360,7 +394,7 @@ template <typename Char> struct ansi_color_escape {
for (int i = 0; i < 7; i++) {
buffer[i] = static_cast<Char>(esc[i]);
}
rgb color(text_color.value.rgb_color);
rgb color(text_color.value());
to_esc(color.r, buffer + 7, ';');
to_esc(color.g, buffer + 11, ';');
to_esc(color.b, buffer + 15, 'm');
@@ -441,32 +475,26 @@ template <typename T> struct styled_arg : view {
};
template <typename Char>
void vformat_to(buffer<Char>& buf, const text_style& ts,
basic_string_view<Char> fmt,
void vformat_to(buffer<Char>& buf, text_style ts, basic_string_view<Char> fmt,
basic_format_args<buffered_context<Char>> args) {
bool has_style = false;
if (ts.has_emphasis()) {
has_style = true;
auto emphasis = make_emphasis<Char>(ts.get_emphasis());
buf.append(emphasis.begin(), emphasis.end());
}
if (ts.has_foreground()) {
has_style = true;
auto foreground = make_foreground_color<Char>(ts.get_foreground());
buf.append(foreground.begin(), foreground.end());
}
if (ts.has_background()) {
has_style = true;
auto background = make_background_color<Char>(ts.get_background());
buf.append(background.begin(), background.end());
}
vformat_to(buf, fmt, args);
if (has_style) reset_color<Char>(buf);
if (ts != text_style()) reset_color<Char>(buf);
}
} // namespace detail
inline void vprint(FILE* f, const text_style& ts, string_view fmt,
format_args args) {
inline void vprint(FILE* f, text_style ts, string_view fmt, format_args args) {
auto buf = memory_buffer();
detail::vformat_to(buf, ts, fmt, args);
print(f, FMT_STRING("{}"), string_view(buf.begin(), buf.size()));
@@ -482,8 +510,7 @@ inline void vprint(FILE* f, const text_style& ts, string_view fmt,
* "Elapsed time: {0:.2f} seconds", 1.23);
*/
template <typename... T>
void print(FILE* f, const text_style& ts, format_string<T...> fmt,
T&&... args) {
void print(FILE* f, text_style ts, format_string<T...> fmt, T&&... args) {
vprint(f, ts, fmt.str, vargs<T...>{{args...}});
}
@@ -497,11 +524,11 @@ void print(FILE* f, const text_style& ts, format_string<T...> fmt,
* "Elapsed time: {0:.2f} seconds", 1.23);
*/
template <typename... T>
void print(const text_style& ts, format_string<T...> fmt, T&&... args) {
void print(text_style ts, format_string<T...> fmt, T&&... args) {
return print(stdout, ts, fmt, std::forward<T>(args)...);
}
inline auto vformat(const text_style& ts, string_view fmt, format_args args)
inline auto vformat(text_style ts, string_view fmt, format_args args)
-> std::string {
auto buf = memory_buffer();
detail::vformat_to(buf, ts, fmt, args);
@@ -521,7 +548,7 @@ inline auto vformat(const text_style& ts, string_view fmt, format_args args)
* ```
*/
template <typename... T>
inline auto format(const text_style& ts, format_string<T...> fmt, T&&... args)
inline auto format(text_style ts, format_string<T...> fmt, T&&... args)
-> std::string {
return fmt::vformat(ts, fmt.str, vargs<T...>{{args...}});
}
@@ -529,8 +556,8 @@ inline auto format(const text_style& ts, format_string<T...> fmt, T&&... args)
/// Formats a string with the given text_style and writes the output to `out`.
template <typename OutputIt,
FMT_ENABLE_IF(detail::is_output_iterator<OutputIt, char>::value)>
auto vformat_to(OutputIt out, const text_style& ts, string_view fmt,
format_args args) -> OutputIt {
auto vformat_to(OutputIt out, text_style ts, string_view fmt, format_args args)
-> OutputIt {
auto&& buf = detail::get_buffer<char>(out);
detail::vformat_to(buf, ts, fmt, args);
return detail::get_iterator(buf, out);
@@ -548,8 +575,8 @@ auto vformat_to(OutputIt out, const text_style& ts, string_view fmt,
*/
template <typename OutputIt, typename... T,
FMT_ENABLE_IF(detail::is_output_iterator<OutputIt, char>::value)>
inline auto format_to(OutputIt out, const text_style& ts,
format_string<T...> fmt, T&&... args) -> OutputIt {
inline auto format_to(OutputIt out, text_style ts, format_string<T...> fmt,
T&&... args) -> OutputIt {
return vformat_to(out, ts, fmt.str, vargs<T...>{{args...}});
}

View File

@@ -19,11 +19,11 @@ FMT_BEGIN_NAMESPACE
// A compile-time string which is compiled into fast formatting code.
FMT_EXPORT class compiled_string {};
namespace detail {
template <typename S>
struct is_compiled_string : std::is_base_of<compiled_string, S> {};
namespace detail {
/**
* Converts a string literal `s` into a format string that will be parsed at
* compile time and converted into efficient formatting code. Requires C++17
@@ -41,16 +41,6 @@ struct is_compiled_string : std::is_base_of<compiled_string, S> {};
# define FMT_COMPILE(s) FMT_STRING(s)
#endif
#if FMT_USE_NONTYPE_TEMPLATE_ARGS
template <typename Char, size_t N, fmt::detail::fixed_string<Char, N> Str>
struct udl_compiled_string : compiled_string {
using char_type = Char;
constexpr explicit operator basic_string_view<char_type>() const {
return {Str.data, N - 1};
}
};
#endif
template <typename T, typename... Tail>
auto first(const T& value, const Tail&...) -> const T& {
return value;
@@ -425,7 +415,7 @@ constexpr auto compile_format_string(S fmt) {
}
template <typename... Args, typename S,
FMT_ENABLE_IF(detail::is_compiled_string<S>::value)>
FMT_ENABLE_IF(is_compiled_string<S>::value)>
constexpr auto compile(S fmt) {
constexpr auto str = basic_string_view<typename S::char_type>(fmt);
if constexpr (str.size() == 0) {
@@ -461,7 +451,7 @@ constexpr FMT_INLINE OutputIt format_to(OutputIt out, const CompiledFormat& cf,
}
template <typename S, typename... Args,
FMT_ENABLE_IF(detail::is_compiled_string<S>::value)>
FMT_ENABLE_IF(is_compiled_string<S>::value)>
FMT_INLINE std::basic_string<typename S::char_type> format(const S&,
Args&&... args) {
if constexpr (std::is_same<typename S::char_type, char>::value) {
@@ -488,7 +478,7 @@ FMT_INLINE std::basic_string<typename S::char_type> format(const S&,
}
template <typename OutputIt, typename S, typename... Args,
FMT_ENABLE_IF(detail::is_compiled_string<S>::value)>
FMT_ENABLE_IF(is_compiled_string<S>::value)>
FMT_CONSTEXPR OutputIt format_to(OutputIt out, const S&, Args&&... args) {
constexpr auto compiled = detail::compile<Args...>(S());
if constexpr (std::is_same<remove_cvref_t<decltype(compiled)>,
@@ -503,7 +493,7 @@ FMT_CONSTEXPR OutputIt format_to(OutputIt out, const S&, Args&&... args) {
#endif
template <typename OutputIt, typename S, typename... Args,
FMT_ENABLE_IF(detail::is_compiled_string<S>::value)>
FMT_ENABLE_IF(is_compiled_string<S>::value)>
auto format_to_n(OutputIt out, size_t n, const S& fmt, Args&&... args)
-> format_to_n_result<OutputIt> {
using traits = detail::fixed_buffer_traits;
@@ -513,7 +503,7 @@ auto format_to_n(OutputIt out, size_t n, const S& fmt, Args&&... args)
}
template <typename S, typename... Args,
FMT_ENABLE_IF(detail::is_compiled_string<S>::value)>
FMT_ENABLE_IF(is_compiled_string<S>::value)>
FMT_CONSTEXPR20 auto formatted_size(const S& fmt, const Args&... args)
-> size_t {
auto buf = detail::counting_buffer<>();
@@ -522,7 +512,7 @@ FMT_CONSTEXPR20 auto formatted_size(const S& fmt, const Args&... args)
}
template <typename S, typename... Args,
FMT_ENABLE_IF(detail::is_compiled_string<S>::value)>
FMT_ENABLE_IF(is_compiled_string<S>::value)>
void print(std::FILE* f, const S& fmt, const Args&... args) {
auto buf = memory_buffer();
fmt::format_to(appender(buf), fmt, args...);
@@ -530,7 +520,7 @@ void print(std::FILE* f, const S& fmt, const Args&... args) {
}
template <typename S, typename... Args,
FMT_ENABLE_IF(detail::is_compiled_string<S>::value)>
FMT_ENABLE_IF(is_compiled_string<S>::value)>
void print(const S& fmt, const Args&... args) {
print(stdout, fmt, args...);
}
@@ -538,9 +528,7 @@ void print(const S& fmt, const Args&... args) {
#if FMT_USE_NONTYPE_TEMPLATE_ARGS
inline namespace literals {
template <detail::fixed_string Str> constexpr auto operator""_cf() {
using char_t = remove_cvref_t<decltype(Str.data[0])>;
return detail::udl_compiled_string<char_t, sizeof(Str.data) / sizeof(char_t),
Str>();
return FMT_COMPILE(Str.data);
}
} // namespace literals
#endif

View File

@@ -212,7 +212,7 @@ inline auto floor_log10_pow2_minus_log10_4_over_3(int e) noexcept -> int {
return (e * 631305 - 261663) >> 21;
}
FMT_INLINE_VARIABLE constexpr struct {
FMT_INLINE_VARIABLE constexpr struct div_small_pow10_infos_struct {
uint32_t divisor;
int shift_amount;
} div_small_pow10_infos[] = {{10, 16}, {100, 16}};
@@ -1097,7 +1097,7 @@ template <> struct cache_accessor<double> {
return {r.high(), r.low() == 0};
}
static auto compute_delta(cache_entry_type const& cache, int beta) noexcept
static auto compute_delta(const cache_entry_type& cache, int beta) noexcept
-> uint32_t {
return static_cast<uint32_t>(cache.high() >> (64 - 1 - beta));
}
@@ -1526,9 +1526,8 @@ template <typename F> class glibc_file : public file_base<F> {
}
void init_buffer() {
if (this->file_->_IO_write_ptr) return;
if (this->file_->_IO_write_ptr < this->file_->_IO_write_end) return;
// Force buffer initialization by placing and removing a char in a buffer.
assume(this->file_->_IO_write_ptr >= this->file_->_IO_write_end);
putc_unlocked(0, this->file_);
--this->file_->_IO_write_ptr;
}

View File

@@ -117,6 +117,7 @@
# define FMT_NOINLINE
#endif
// GCC 4.9 doesn't support qualified names in specializations.
namespace std {
template <typename T> struct iterator_traits<fmt::basic_appender<T>> {
using iterator_category = output_iterator_tag;
@@ -227,7 +228,9 @@ FMT_CONSTEXPR inline void abort_fuzzing_if(bool condition) {
#if defined(FMT_USE_STRING_VIEW)
template <typename Char> using std_string_view = std::basic_string_view<Char>;
#else
template <typename T> struct std_string_view {};
template <typename Char> struct std_string_view {
operator basic_string_view<Char>() const;
};
#endif
template <typename Char, Char... C> struct string_literal {
@@ -703,7 +706,7 @@ using is_integer =
#if defined(FMT_USE_FLOAT128)
// Use the provided definition.
#elif FMT_CLANG_VERSION && FMT_HAS_INCLUDE(<quadmath.h>)
#elif FMT_CLANG_VERSION >= 309 && FMT_HAS_INCLUDE(<quadmath.h>)
# define FMT_USE_FLOAT128 1
#elif FMT_GCC_VERSION && defined(_GLIBCXX_USE_FLOAT128) && \
!defined(__STRICT_ANSI__)
@@ -719,11 +722,10 @@ struct float128 {};
template <typename T> using is_float128 = std::is_same<T, float128>;
template <typename T>
using is_floating_point =
bool_constant<std::is_floating_point<T>::value || is_float128<T>::value>;
template <typename T> struct is_floating_point : std::is_floating_point<T> {};
template <> struct is_floating_point<float128> : std::true_type {};
template <typename T, bool = std::is_floating_point<T>::value>
template <typename T, bool = is_floating_point<T>::value>
struct is_fast_float : bool_constant<std::numeric_limits<T>::is_iec559 &&
sizeof(T) <= sizeof(double)> {};
template <typename T> struct is_fast_float<T, false> : std::false_type {};
@@ -1203,7 +1205,7 @@ FMT_CONSTEXPR FMT_INLINE auto format_decimal(Char* out, UInt value,
}
template <typename Char, typename UInt, typename OutputIt,
FMT_ENABLE_IF(is_back_insert_iterator<OutputIt>::value)>
FMT_ENABLE_IF(!std::is_pointer<remove_cvref_t<OutputIt>>::value)>
FMT_CONSTEXPR auto format_decimal(OutputIt out, UInt value, int num_digits)
-> OutputIt {
if (auto ptr = to_pointer<Char>(out, to_unsigned(num_digits))) {
@@ -1611,7 +1613,7 @@ constexpr auto convert_float(T value) -> convert_float_result<T> {
}
template <typename Char, typename OutputIt>
FMT_NOINLINE FMT_CONSTEXPR auto fill(OutputIt it, size_t n,
FMT_CONSTEXPR FMT_NOINLINE auto fill(OutputIt it, size_t n,
const basic_specs& specs) -> OutputIt {
auto fill_size = specs.fill_size();
if (fill_size == 1) return detail::fill_n(it, n, specs.fill_unit<Char>());
@@ -2330,7 +2332,7 @@ template <typename Char, typename OutputIt, typename DecimalFP,
typename Grouping = digit_grouping<Char>>
FMT_CONSTEXPR20 auto do_write_float(OutputIt out, const DecimalFP& f,
const format_specs& specs, sign s,
locale_ref loc) -> OutputIt {
int exp_upper, locale_ref loc) -> OutputIt {
auto significand = f.significand;
int significand_size = get_significand_size(f);
const Char zero = static_cast<Char>('0');
@@ -2346,7 +2348,7 @@ FMT_CONSTEXPR20 auto do_write_float(OutputIt out, const DecimalFP& f,
if (specs.type() == presentation_type::fixed) return false;
// Use the fixed notation if the exponent is in [exp_lower, exp_upper),
// e.g. 0.0001 instead of 1e-04. Otherwise use the exponent notation.
const int exp_lower = -4, exp_upper = 16;
const int exp_lower = -4;
return output_exp < exp_lower ||
output_exp >= (specs.precision > 0 ? specs.precision : exp_upper);
};
@@ -2449,12 +2451,13 @@ template <typename Char> class fallback_digit_grouping {
template <typename Char, typename OutputIt, typename DecimalFP>
FMT_CONSTEXPR20 auto write_float(OutputIt out, const DecimalFP& f,
const format_specs& specs, sign s,
locale_ref loc) -> OutputIt {
int exp_upper, locale_ref loc) -> OutputIt {
if (is_constant_evaluated()) {
return do_write_float<Char, OutputIt, DecimalFP,
fallback_digit_grouping<Char>>(out, f, specs, s, loc);
fallback_digit_grouping<Char>>(out, f, specs, s,
exp_upper, loc);
} else {
return do_write_float<Char>(out, f, specs, s, loc);
return do_write_float<Char>(out, f, specs, s, exp_upper, loc);
}
}
@@ -2469,8 +2472,8 @@ template <typename T>
struct has_isfinite<T, enable_if_t<sizeof(std::isfinite(T())) != 0>>
: std::true_type {};
template <typename T, FMT_ENABLE_IF(std::is_floating_point<T>::value&&
has_isfinite<T>::value)>
template <typename T,
FMT_ENABLE_IF(is_floating_point<T>::value&& has_isfinite<T>::value)>
FMT_CONSTEXPR20 auto isfinite(T value) -> bool {
constexpr T inf = T(std::numeric_limits<double>::infinity());
if (is_constant_evaluated())
@@ -3286,6 +3289,14 @@ FMT_CONSTEXPR20 auto format_float(Float value, int precision,
return exp;
}
// Numbers with exponents greater or equal to the returned value will use
// the exponential notation.
template <typename T> constexpr auto exp_upper() -> int {
return std::numeric_limits<T>::digits10 != 0
? min_of(16, std::numeric_limits<T>::digits10 + 1)
: 16;
}
template <typename Char, typename OutputIt, typename T>
FMT_CONSTEXPR20 auto write_float(OutputIt out, T value, format_specs specs,
locale_ref loc) -> OutputIt {
@@ -3301,6 +3312,7 @@ FMT_CONSTEXPR20 auto write_float(OutputIt out, T value, format_specs specs,
if (specs.width != 0) --specs.width;
}
constexpr int exp_upper = detail::exp_upper<T>();
int precision = specs.precision;
if (precision < 0) {
if (specs.type() != presentation_type::none) {
@@ -3309,7 +3321,7 @@ FMT_CONSTEXPR20 auto write_float(OutputIt out, T value, format_specs specs,
// Use Dragonbox for the shortest format.
using floaty = conditional_t<sizeof(T) >= sizeof(double), double, float>;
auto dec = dragonbox::to_decimal(static_cast<floaty>(value));
return write_float<Char>(out, dec, specs, s, loc);
return write_float<Char>(out, dec, specs, s, exp_upper, loc);
}
}
@@ -3337,7 +3349,7 @@ FMT_CONSTEXPR20 auto write_float(OutputIt out, T value, format_specs specs,
specs.precision = precision;
auto f = big_decimal_fp{buffer.data(), static_cast<int>(buffer.size()), exp};
return write_float<Char>(out, f, specs, s, loc);
return write_float<Char>(out, f, specs, s, exp_upper, loc);
}
template <typename Char, typename OutputIt, typename T,
@@ -3364,7 +3376,7 @@ FMT_CONSTEXPR20 auto write(OutputIt out, T value) -> OutputIt {
return write_nonfinite<Char>(out, std::isnan(value), specs, s);
auto dec = dragonbox::to_decimal(static_cast<floaty>(value));
return write_float<Char>(out, dec, specs, s, {});
return write_float<Char>(out, dec, specs, s, exp_upper<T>(), {});
}
template <typename Char, typename OutputIt, typename T,

View File

@@ -150,7 +150,7 @@ inline void vprint(std::ostream& os, string_view fmt, format_args args) {
FMT_EXPORT template <typename... T>
void print(std::ostream& os, format_string<T...> fmt, T&&... args) {
fmt::vargs<T...> vargs = {{args...}};
if (detail::use_utf8) return vprint(os, fmt.str, vargs);
if (detail::const_check(detail::use_utf8)) return vprint(os, fmt.str, vargs);
auto buffer = memory_buffer();
detail::vformat_to(buffer, fmt.str, vargs);
detail::write_buffer(os, buffer);
@@ -158,7 +158,8 @@ void print(std::ostream& os, format_string<T...> fmt, T&&... args) {
FMT_EXPORT template <typename... T>
void println(std::ostream& os, format_string<T...> fmt, T&&... args) {
fmt::print(os, "{}\n", fmt::format(fmt, std::forward<T>(args)...));
fmt::print(os, FMT_STRING("{}\n"),
fmt::format(fmt, std::forward<T>(args)...));
}
FMT_END_NAMESPACE

View File

@@ -527,7 +527,9 @@ struct formatter<
template <typename R, typename Char>
struct formatter<
R, Char,
enable_if_t<range_format_kind<R, Char>::value == range_format::map>> {
enable_if_t<conjunction<
bool_constant<range_format_kind<R, Char>::value == range_format::map>,
detail::is_formattable_delayed<R, Char>>::value>> {
private:
using map_type = detail::maybe_const_range<R>;
using element_type = detail::uncvref_type<map_type>;
@@ -772,13 +774,13 @@ struct formatter<
: formatter<detail::all<typename T::container_type>, Char> {
using all = detail::all<typename T::container_type>;
template <typename FormatContext>
auto format(const T& t, FormatContext& ctx) const -> decltype(ctx.out()) {
auto format(const T& value, FormatContext& ctx) const -> decltype(ctx.out()) {
struct getter : T {
static auto get(const T& t) -> all {
return {t.*(&getter::c)}; // Access c through the derived class.
static auto get(const T& v) -> all {
return {v.*(&getter::c)}; // Access c through the derived class.
}
};
return formatter<all>::format(getter::get(t), ctx);
return formatter<all>::format(getter::get(value), ctx);
}
};

View File

@@ -113,7 +113,6 @@ void write_escaped_path(basic_memory_buffer<Char>& quoted,
} // namespace detail
FMT_EXPORT
template <typename Char> struct formatter<std::filesystem::path, Char> {
private:
format_specs specs_;
@@ -182,7 +181,6 @@ FMT_END_NAMESPACE
#endif // FMT_CPP_LIB_FILESYSTEM
FMT_BEGIN_NAMESPACE
FMT_EXPORT
template <std::size_t N, typename Char>
struct formatter<std::bitset<N>, Char>
: nested_formatter<basic_string_view<Char>, Char> {
@@ -209,14 +207,12 @@ struct formatter<std::bitset<N>, Char>
}
};
FMT_EXPORT
template <typename Char>
struct formatter<std::thread::id, Char> : basic_ostream_formatter<Char> {};
FMT_END_NAMESPACE
#ifdef __cpp_lib_optional
FMT_BEGIN_NAMESPACE
FMT_EXPORT
template <typename T, typename Char>
struct formatter<std::optional<T>, Char,
std::enable_if_t<is_formattable<T, Char>::value>> {
@@ -279,7 +275,6 @@ FMT_END_NAMESPACE
#ifdef __cpp_lib_expected
FMT_BEGIN_NAMESPACE
FMT_EXPORT
template <typename T, typename E, typename Char>
struct formatter<std::expected<T, E>, Char,
std::enable_if_t<(std::is_void<T>::value ||
@@ -311,7 +306,6 @@ FMT_END_NAMESPACE
#ifdef __cpp_lib_source_location
FMT_BEGIN_NAMESPACE
FMT_EXPORT
template <> struct formatter<std::source_location> {
FMT_CONSTEXPR auto parse(parse_context<>& ctx) { return ctx.begin(); }
@@ -367,7 +361,6 @@ template <typename T, typename C> struct is_variant_formattable {
detail::is_variant_formattable_<T, C>::value;
};
FMT_EXPORT
template <typename Char> struct formatter<std::monostate, Char> {
FMT_CONSTEXPR auto parse(parse_context<Char>& ctx) -> const Char* {
return ctx.begin();
@@ -380,7 +373,6 @@ template <typename Char> struct formatter<std::monostate, Char> {
}
};
FMT_EXPORT
template <typename Variant, typename Char>
struct formatter<
Variant, Char,
@@ -414,11 +406,11 @@ FMT_END_NAMESPACE
#endif // FMT_CPP_LIB_VARIANT
FMT_BEGIN_NAMESPACE
FMT_EXPORT
template <> struct formatter<std::error_code> {
private:
format_specs specs_;
detail::arg_ref<char> width_ref_;
bool debug_ = false;
public:
FMT_CONSTEXPR auto parse(parse_context<>& ctx) -> const char* {
@@ -426,11 +418,19 @@ template <> struct formatter<std::error_code> {
if (it == end) return it;
it = detail::parse_align(it, end, specs_);
if (it == end) return it;
char c = *it;
if ((c >= '0' && c <= '9') || c == '{')
if (it != end && ((c >= '0' && c <= '9') || c == '{'))
it = detail::parse_width(it, end, specs_, width_ref_, ctx);
if (it != end && *it == '?') {
debug_ = true;
++it;
}
if (it != end && *it == 's') {
specs_.set_type(presentation_type::string);
++it;
}
return it;
}
@@ -440,12 +440,21 @@ template <> struct formatter<std::error_code> {
auto specs = specs_;
detail::handle_dynamic_spec(specs.dynamic_width(), specs.width, width_ref_,
ctx);
memory_buffer buf;
buf.append(string_view(ec.category().name()));
buf.push_back(':');
detail::write<char>(appender(buf), ec.value());
return detail::write<char>(ctx.out(), string_view(buf.data(), buf.size()),
specs);
auto buf = memory_buffer();
if (specs_.type() == presentation_type::string) {
buf.append(ec.message());
} else {
buf.append(string_view(ec.category().name()));
buf.push_back(':');
detail::write<char>(appender(buf), ec.value());
}
auto quoted = memory_buffer();
auto str = string_view(buf.data(), buf.size());
if (debug_) {
detail::write_escaped_string<char>(std::back_inserter(quoted), str);
str = string_view(quoted.data(), quoted.size());
}
return detail::write<char>(ctx.out(), str, specs);
}
};
@@ -520,7 +529,6 @@ auto write_demangled_name(OutputIt out, const std::type_info& ti) -> OutputIt {
} // namespace detail
FMT_EXPORT
template <typename Char>
struct formatter<std::type_info, Char // DEPRECATED! Mixing code unit types.
> {
@@ -537,7 +545,6 @@ struct formatter<std::type_info, Char // DEPRECATED! Mixing code unit types.
};
#endif
FMT_EXPORT
template <typename T, typename Char>
struct formatter<
T, Char, // DEPRECATED! Mixing code unit types.
@@ -603,7 +610,6 @@ struct is_bit_reference_like<std::__bit_const_reference<C>> {
// We can't use std::vector<bool, Allocator>::reference and
// std::bitset<N>::reference because the compiler can't deduce Allocator and N
// in partial specialization.
FMT_EXPORT
template <typename BitRef, typename Char>
struct formatter<BitRef, Char,
enable_if_t<detail::is_bit_reference_like<BitRef>::value>>
@@ -623,7 +629,6 @@ template <typename T> auto ptr(const std::shared_ptr<T>& p) -> const void* {
return p.get();
}
FMT_EXPORT
template <typename T, typename Char>
struct formatter<std::atomic<T>, Char,
enable_if_t<is_formattable<T, Char>::value>>
@@ -636,7 +641,6 @@ struct formatter<std::atomic<T>, Char,
};
#ifdef __cpp_lib_atomic_flag_test
FMT_EXPORT
template <typename Char>
struct formatter<std::atomic_flag, Char> : formatter<bool, Char> {
template <typename FormatContext>
@@ -647,7 +651,6 @@ struct formatter<std::atomic_flag, Char> : formatter<bool, Char> {
};
#endif // __cpp_lib_atomic_flag_test
FMT_EXPORT
template <typename T, typename Char> struct formatter<std::complex<T>, Char> {
private:
detail::dynamic_format_specs<Char> specs_;
@@ -710,7 +713,6 @@ template <typename T, typename Char> struct formatter<std::complex<T>, Char> {
}
};
FMT_EXPORT
template <typename T, typename Char>
struct formatter<std::reference_wrapper<T>, Char,
enable_if_t<is_formattable<remove_cvref_t<T>, Char>::value>>

View File

@@ -112,10 +112,6 @@ inline auto runtime(wstring_view s) -> runtime_format_string<wchar_t> {
return {{s}};
}
template <> struct is_char<wchar_t> : std::true_type {};
template <> struct is_char<char16_t> : std::true_type {};
template <> struct is_char<char32_t> : std::true_type {};
#ifdef __cpp_char8_t
template <> struct is_char<char8_t> : bool_constant<detail::is_utf8_enabled> {};
#endif
@@ -322,7 +318,7 @@ template <typename... T> void println(wformat_string<T...> fmt, T&&... args) {
return print(L"{}\n", fmt::format(fmt, std::forward<T>(args)...));
}
inline auto vformat(const text_style& ts, wstring_view fmt, wformat_args args)
inline auto vformat(text_style ts, wstring_view fmt, wformat_args args)
-> std::wstring {
auto buf = wmemory_buffer();
detail::vformat_to(buf, ts, fmt, args);
@@ -330,19 +326,19 @@ inline auto vformat(const text_style& ts, wstring_view fmt, wformat_args args)
}
template <typename... T>
inline auto format(const text_style& ts, wformat_string<T...> fmt, T&&... args)
inline auto format(text_style ts, wformat_string<T...> fmt, T&&... args)
-> std::wstring {
return fmt::vformat(ts, fmt, fmt::make_wformat_args(args...));
}
template <typename... T>
FMT_DEPRECATED void print(std::FILE* f, const text_style& ts,
wformat_string<T...> fmt, const T&... args) {
FMT_DEPRECATED void print(std::FILE* f, text_style ts, wformat_string<T...> fmt,
const T&... args) {
vprint(f, ts, fmt, fmt::make_wformat_args(args...));
}
template <typename... T>
FMT_DEPRECATED void print(const text_style& ts, wformat_string<T...> fmt,
FMT_DEPRECATED void print(text_style ts, wformat_string<T...> fmt,
const T&... args) {
return print(stdout, ts, fmt, args...);
}

View File

@@ -1,5 +1,7 @@
module;
#define FMT_MODULE
#ifdef _MSVC_LANG
# define FMT_CPLUSPLUS _MSVC_LANG
#else

View File

@@ -9,7 +9,7 @@ GoogleTest now follows the
We recommend
[updating to the latest commit in the `main` branch as often as possible](https://github.com/abseil/abseil-cpp/blob/master/FAQ.md#what-is-live-at-head-and-how-do-i-do-it).
We do publish occasional semantic versions, tagged with
`v${major}.${minor}.${patch}` (e.g. `v1.15.0`).
`v${major}.${minor}.${patch}` (e.g. `v1.16.0`).
#### Documentation Updates
@@ -17,12 +17,12 @@ Our documentation is now live on GitHub Pages at
https://google.github.io/googletest/. We recommend browsing the documentation on
GitHub Pages rather than directly in the repository.
#### Release 1.15.0
#### Release 1.16.0
[Release 1.15.0](https://github.com/google/googletest/releases/tag/v1.15.0) is
[Release 1.16.0](https://github.com/google/googletest/releases/tag/v1.16.0) is
now available.
The 1.15.x branch requires at least C++14.
The 1.16.x branch requires at least C++14.
#### Continuous Integration

View File

@@ -1493,6 +1493,7 @@ class DoAllAction<FinalAction> {
// providing a call operator because even with a particular set of arguments
// they don't have a fixed return type.
// We support conversion to OnceAction whenever the sub-action does.
template <typename R, typename... Args,
typename std::enable_if<
std::is_convertible<FinalAction, OnceAction<R(Args...)>>::value,
@@ -1501,6 +1502,21 @@ class DoAllAction<FinalAction> {
return std::move(final_action_);
}
// We also support conversion to OnceAction whenever the sub-action supports
// conversion to Action (since any Action can also be a OnceAction).
template <
typename R, typename... Args,
typename std::enable_if<
conjunction<
negation<
std::is_convertible<FinalAction, OnceAction<R(Args...)>>>,
std::is_convertible<FinalAction, Action<R(Args...)>>>::value,
int>::type = 0>
operator OnceAction<R(Args...)>() && { // NOLINT
return Action<R(Args...)>(std::move(final_action_));
}
// We support conversion to Action whenever the sub-action does.
template <
typename R, typename... Args,
typename std::enable_if<
@@ -1580,16 +1596,16 @@ class DoAllAction<InitialAction, OtherActions...>
: Base({}, std::forward<U>(other_actions)...),
initial_action_(std::forward<T>(initial_action)) {}
template <typename R, typename... Args,
typename std::enable_if<
conjunction<
// Both the initial action and the rest must support
// conversion to OnceAction.
std::is_convertible<
InitialAction,
OnceAction<void(InitialActionArgType<Args>...)>>,
std::is_convertible<Base, OnceAction<R(Args...)>>>::value,
int>::type = 0>
// We support conversion to OnceAction whenever both the initial action and
// the rest support conversion to OnceAction.
template <
typename R, typename... Args,
typename std::enable_if<
conjunction<std::is_convertible<
InitialAction,
OnceAction<void(InitialActionArgType<Args>...)>>,
std::is_convertible<Base, OnceAction<R(Args...)>>>::value,
int>::type = 0>
operator OnceAction<R(Args...)>() && { // NOLINT
// Return an action that first calls the initial action with arguments
// filtered through InitialActionArgType, then forwards arguments directly
@@ -1612,12 +1628,34 @@ class DoAllAction<InitialAction, OtherActions...>
};
}
// We also support conversion to OnceAction whenever the initial action
// supports conversion to Action (since any Action can also be a OnceAction).
//
// The remaining sub-actions must also be compatible, but we don't need to
// special case them because the base class deals with them.
template <
typename R, typename... Args,
typename std::enable_if<
conjunction<
negation<std::is_convertible<
InitialAction,
OnceAction<void(InitialActionArgType<Args>...)>>>,
std::is_convertible<InitialAction,
Action<void(InitialActionArgType<Args>...)>>,
std::is_convertible<Base, OnceAction<R(Args...)>>>::value,
int>::type = 0>
operator OnceAction<R(Args...)>() && { // NOLINT
return DoAll(
Action<void(InitialActionArgType<Args>...)>(std::move(initial_action_)),
std::move(static_cast<Base&>(*this)));
}
// We support conversion to Action whenever both the initial action and the
// rest support conversion to Action.
template <
typename R, typename... Args,
typename std::enable_if<
conjunction<
// Both the initial action and the rest must support conversion to
// Action.
std::is_convertible<const InitialAction&,
Action<void(InitialActionArgType<Args>...)>>,
std::is_convertible<const Base&, Action<R(Args...)>>>::value,
@@ -1665,8 +1703,9 @@ template <size_t k>
struct ReturnArgAction {
template <typename... Args,
typename = typename std::enable_if<(k < sizeof...(Args))>::type>
auto operator()(Args&&... args) const -> decltype(std::get<k>(
std::forward_as_tuple(std::forward<Args>(args)...))) {
auto operator()(Args&&... args) const
-> decltype(std::get<k>(
std::forward_as_tuple(std::forward<Args>(args)...))) {
return std::get<k>(std::forward_as_tuple(std::forward<Args>(args)...));
}
};

View File

@@ -408,13 +408,22 @@ class MatcherCastImpl<T, Matcher<U>> {
}
private:
class Impl : public MatcherInterface<T> {
// If it's possible to implicitly convert a `const T&` to U, then `Impl` can
// take that as input to avoid a copy. Otherwise, such as when `T` is a
// non-const reference type or a type explicitly constructible only from a
// non-const reference, then `Impl` must use `T` as-is (potentially copying).
using ImplArgT =
typename std::conditional<std::is_convertible<const T&, const U&>::value,
const T&, T>::type;
class Impl : public MatcherInterface<ImplArgT> {
public:
explicit Impl(const Matcher<U>& source_matcher)
: source_matcher_(source_matcher) {}
// We delegate the matching logic to the source matcher.
bool MatchAndExplain(T x, MatchResultListener* listener) const override {
bool MatchAndExplain(ImplArgT x,
MatchResultListener* listener) const override {
using FromType = typename std::remove_cv<typename std::remove_pointer<
typename std::remove_reference<T>::type>::type>::type;
using ToType = typename std::remove_cv<typename std::remove_pointer<
@@ -431,9 +440,8 @@ class MatcherCastImpl<T, Matcher<U>> {
// Do the cast to `U` explicitly if necessary.
// Otherwise, let implicit conversions do the trick.
using CastType =
typename std::conditional<std::is_convertible<T&, const U&>::value,
T&, U>::type;
using CastType = typename std::conditional<
std::is_convertible<ImplArgT&, const U&>::value, ImplArgT&, U>::type;
return source_matcher_.MatchAndExplain(static_cast<CastType>(x),
listener);
@@ -528,18 +536,16 @@ inline Matcher<T> SafeMatcherCast(const M& polymorphic_matcher_or_value) {
// safely convert a Matcher<U> to a Matcher<T> (i.e. Matcher is
// contravariant): just keep a copy of the original Matcher<U>, convert the
// argument from type T to U, and then pass it to the underlying Matcher<U>.
// The only exception is when U is a reference and T is not, as the
// The only exception is when U is a non-const reference and T is not, as the
// underlying Matcher<U> may be interested in the argument's address, which
// is not preserved in the conversion from T to U.
// cannot be preserved in the conversion from T to U (since a copy of the input
// T argument would be required to provide a non-const reference U).
template <typename T, typename U>
inline Matcher<T> SafeMatcherCast(const Matcher<U>& matcher) {
// Enforce that T can be implicitly converted to U.
static_assert(std::is_convertible<const T&, const U&>::value,
"T must be implicitly convertible to U");
// Enforce that we are not converting a non-reference type T to a reference
// type U.
static_assert(std::is_reference<T>::value || !std::is_reference<U>::value,
"cannot convert non reference arg to reference");
"T must be implicitly convertible to U (and T must be a "
"non-const reference if U is a non-const reference)");
// In case both T and U are arithmetic types, enforce that the
// conversion is not lossy.
typedef GTEST_REMOVE_REFERENCE_AND_CONST_(T) RawT;
@@ -561,6 +567,11 @@ Matcher<T> A();
// and MUST NOT BE USED IN USER CODE!!!
namespace internal {
// Used per go/ranked-overloads for dispatching.
struct Rank0 {};
struct Rank1 : Rank0 {};
using HighestRank = Rank1;
// If the explanation is not empty, prints it to the ostream.
inline void PrintIfNotEmpty(const std::string& explanation,
::std::ostream* os) {
@@ -1300,34 +1311,48 @@ class AllOfMatcherImpl : public MatcherInterface<const T&> {
bool MatchAndExplain(const T& x,
MatchResultListener* listener) const override {
// If either matcher1_ or matcher2_ doesn't match x, we only need
// to explain why one of them fails.
// This method uses matcher's explanation when explaining the result.
// However, if matcher doesn't provide one, this method uses matcher's
// description.
std::string all_match_result;
for (size_t i = 0; i < matchers_.size(); ++i) {
for (const Matcher<T>& matcher : matchers_) {
StringMatchResultListener slistener;
if (matchers_[i].MatchAndExplain(x, &slistener)) {
if (all_match_result.empty()) {
all_match_result = slistener.str();
// Return explanation for first failed matcher.
if (!matcher.MatchAndExplain(x, &slistener)) {
const std::string explanation = slistener.str();
if (!explanation.empty()) {
*listener << explanation;
} else {
std::string result = slistener.str();
if (!result.empty()) {
all_match_result += ", and ";
all_match_result += result;
}
*listener << "which doesn't match (" << Describe(matcher) << ")";
}
} else {
*listener << slistener.str();
return false;
}
// Keep track of explanations in case all matchers succeed.
std::string explanation = slistener.str();
if (explanation.empty()) {
explanation = Describe(matcher);
}
if (all_match_result.empty()) {
all_match_result = explanation;
} else {
if (!explanation.empty()) {
all_match_result += ", and ";
all_match_result += explanation;
}
}
}
// Otherwise we need to explain why *both* of them match.
*listener << all_match_result;
return true;
}
private:
// Returns matcher description as a string.
std::string Describe(const Matcher<T>& matcher) const {
StringMatchResultListener listener;
matcher.DescribeTo(listener.stream());
return listener.str();
}
const std::vector<Matcher<T>> matchers_;
};
@@ -1405,34 +1430,55 @@ class AnyOfMatcherImpl : public MatcherInterface<const T&> {
bool MatchAndExplain(const T& x,
MatchResultListener* listener) const override {
// This method uses matcher's explanation when explaining the result.
// However, if matcher doesn't provide one, this method uses matcher's
// description.
std::string no_match_result;
// If either matcher1_ or matcher2_ matches x, we just need to
// explain why *one* of them matches.
for (size_t i = 0; i < matchers_.size(); ++i) {
for (const Matcher<T>& matcher : matchers_) {
StringMatchResultListener slistener;
if (matchers_[i].MatchAndExplain(x, &slistener)) {
*listener << slistener.str();
return true;
} else {
if (no_match_result.empty()) {
no_match_result = slistener.str();
// Return explanation for first match.
if (matcher.MatchAndExplain(x, &slistener)) {
const std::string explanation = slistener.str();
if (!explanation.empty()) {
*listener << explanation;
} else {
std::string result = slistener.str();
if (!result.empty()) {
no_match_result += ", and ";
no_match_result += result;
}
*listener << "which matches (" << Describe(matcher) << ")";
}
return true;
}
// Keep track of explanations in case there is no match.
std::string explanation = slistener.str();
if (explanation.empty()) {
explanation = DescribeNegation(matcher);
}
if (no_match_result.empty()) {
no_match_result = explanation;
} else {
if (!explanation.empty()) {
no_match_result += ", and ";
no_match_result += explanation;
}
}
}
// Otherwise we need to explain why *both* of them fail.
*listener << no_match_result;
return false;
}
private:
// Returns matcher description as a string.
std::string Describe(const Matcher<T>& matcher) const {
StringMatchResultListener listener;
matcher.DescribeTo(listener.stream());
return listener.str();
}
std::string DescribeNegation(const Matcher<T>& matcher) const {
StringMatchResultListener listener;
matcher.DescribeNegationTo(listener.stream());
return listener.str();
}
const std::vector<Matcher<T>> matchers_;
};
@@ -1483,7 +1529,7 @@ class SomeOfArrayMatcher {
}
private:
const ::std::vector<T> matchers_;
const std::vector<std::remove_const_t<T>> matchers_;
};
template <typename T>
@@ -2235,6 +2281,9 @@ class ResultOfMatcher {
class Impl : public MatcherInterface<T> {
using ResultType = decltype(CallableTraits<Callable>::template Invoke<T>(
std::declval<CallableStorageType>(), std::declval<T>()));
using InnerType = std::conditional_t<
std::is_lvalue_reference<ResultType>::value,
const typename std::remove_reference<ResultType>::type&, ResultType>;
public:
template <typename M>
@@ -2242,7 +2291,7 @@ class ResultOfMatcher {
const CallableStorageType& callable, const M& matcher)
: result_description_(result_description),
callable_(callable),
matcher_(MatcherCast<ResultType>(matcher)) {}
matcher_(MatcherCast<InnerType>(matcher)) {}
void DescribeTo(::std::ostream* os) const override {
if (result_description_.empty()) {
@@ -2272,7 +2321,7 @@ class ResultOfMatcher {
// takes a non-const reference as argument.
// Also, specifying template argument explicitly is needed because T could
// be a non-const reference (e.g. Matcher<Uncopyable&>).
ResultType result =
InnerType result =
CallableTraits<Callable>::template Invoke<T>(callable_, obj);
return MatchPrintAndExplain(result, matcher_, listener);
}
@@ -2285,7 +2334,7 @@ class ResultOfMatcher {
// use stateful callables with ResultOf(), which doesn't guarantee
// how many times the callable will be invoked.
mutable CallableStorageType callable_;
const Matcher<ResultType> matcher_;
const Matcher<InnerType> matcher_;
}; // class Impl
const std::string result_description_;
@@ -2920,10 +2969,6 @@ class EachMatcher {
const M inner_matcher_;
};
// Use go/ranked-overloads for dispatching.
struct Rank0 {};
struct Rank1 : Rank0 {};
namespace pair_getters {
using std::get;
template <typename T>
@@ -3255,6 +3300,11 @@ auto UnpackStructImpl(const T& t, std::make_index_sequence<19>, char) {
const auto& [a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s] = t;
return std::tie(a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s);
}
template <typename T>
auto UnpackStructImpl(const T& u, std::make_index_sequence<20>, char) {
const auto& [a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t] = u;
return std::tie(a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t);
}
#endif // defined(__cpp_structured_bindings)
template <size_t I, typename T>
@@ -3769,7 +3819,7 @@ class UnorderedElementsAreArrayMatcher {
private:
UnorderedMatcherRequire::Flags match_flags_;
::std::vector<T> matchers_;
std::vector<std::remove_const_t<T>> matchers_;
};
// Implements ElementsAreArray().
@@ -3790,7 +3840,7 @@ class ElementsAreArrayMatcher {
}
private:
const ::std::vector<T> matchers_;
const std::vector<std::remove_const_t<T>> matchers_;
};
// Given a 2-tuple matcher tm of type Tuple2Matcher and a value second
@@ -3877,6 +3927,21 @@ GTEST_API_ std::string FormatMatcherDescription(
bool negation, const char* matcher_name,
const std::vector<const char*>& param_names, const Strings& param_values);
// Overloads to support `OptionalMatcher` being used with a type that either
// supports implicit conversion to bool or a `has_value()` method.
template <typename Optional>
auto IsOptionalEngaged(const Optional& optional,
Rank1) -> decltype(!!optional) {
// The use of double-negation here is to preserve historical behavior where
// the matcher used `operator!` rather than directly using `operator bool`.
return !static_cast<bool>(!optional);
}
template <typename Optional>
auto IsOptionalEngaged(const Optional& optional,
Rank0) -> decltype(!optional.has_value()) {
return optional.has_value();
}
// Implements a matcher that checks the value of a optional<> type variable.
template <typename ValueMatcher>
class OptionalMatcher {
@@ -3909,7 +3974,7 @@ class OptionalMatcher {
bool MatchAndExplain(Optional optional,
MatchResultListener* listener) const override {
if (!optional) {
if (!IsOptionalEngaged(optional, HighestRank())) {
*listener << "which is not engaged";
return false;
}
@@ -4742,9 +4807,10 @@ Pointwise(const TupleMatcher& tuple_matcher, const Container& rhs) {
// Supports the Pointwise(m, {a, b, c}) syntax.
template <typename TupleMatcher, typename T>
inline internal::PointwiseMatcher<TupleMatcher, std::vector<T>> Pointwise(
const TupleMatcher& tuple_matcher, std::initializer_list<T> rhs) {
return Pointwise(tuple_matcher, std::vector<T>(rhs));
inline internal::PointwiseMatcher<TupleMatcher,
std::vector<std::remove_const_t<T>>>
Pointwise(const TupleMatcher& tuple_matcher, std::initializer_list<T> rhs) {
return Pointwise(tuple_matcher, std::vector<std::remove_const_t<T>>(rhs));
}
// UnorderedPointwise(pair_matcher, rhs) matches an STL-style
@@ -4906,7 +4972,7 @@ inline internal::UnorderedElementsAreArrayMatcher<T> IsSupersetOf(
// - {1} matches IsSubsetOf({Gt(0), Lt(0)}), as 1 matches Gt(0).
// - {1, -1} matches IsSubsetOf({Lt(0), Gt(0)}), as 1 matches Gt(0) and -1
// matches Lt(0).
// - {1, 2} doesn't matches IsSubsetOf({Gt(0), Lt(0)}), even though 1 and 2 both
// - {1, 2} doesn't match IsSubsetOf({Gt(0), Lt(0)}), even though 1 and 2 both
// match Gt(0). The reason is that different matchers must be used for
// elements in different slots of the container.
//
@@ -5231,9 +5297,10 @@ inline InnerMatcher AllArgs(const InnerMatcher& matcher) {
}
// Returns a matcher that matches the value of an optional<> type variable.
// The matcher implementation only uses '!arg' and requires that the optional<>
// type has a 'value_type' member type and that '*arg' is of type 'value_type'
// and is printable using 'PrintToString'. It is compatible with
// The matcher implementation only uses '!arg' (or 'arg.has_value()' if '!arg`
// isn't a valid expression) and requires that the optional<> type has a
// 'value_type' member type and that '*arg' is of type 'value_type' and is
// printable using 'PrintToString'. It is compatible with
// std::optional/std::experimental::optional.
// Note that to compare an optional type variable against nullopt you should
// use Eq(nullopt) and not Eq(Optional(nullopt)). The latter implies that the

View File

@@ -601,9 +601,10 @@ template <std::size_t index, typename... Params>
struct InvokeArgumentAction {
template <typename... Args,
typename = typename std::enable_if<(index < sizeof...(Args))>::type>
auto operator()(Args &&...args) const -> decltype(internal::InvokeArgument(
std::get<index>(std::forward_as_tuple(std::forward<Args>(args)...)),
std::declval<const Params &>()...)) {
auto operator()(Args &&...args) const
-> decltype(internal::InvokeArgument(
std::get<index>(std::forward_as_tuple(std::forward<Args>(args)...)),
std::declval<const Params &>()...)) {
internal::FlatTuple<Args &&...> args_tuple(FlatTupleConstructTag{},
std::forward<Args>(args)...);
return params.Apply([&](const Params &...unpacked_params) {

View File

@@ -868,7 +868,7 @@ class GTEST_API_ ExpectationBase {
Clause last_clause_;
mutable bool action_count_checked_; // Under mutex_.
mutable Mutex mutex_; // Protects action_count_checked_.
}; // class ExpectationBase
}; // class ExpectationBase
template <typename F>
class TypedExpectation;
@@ -1838,9 +1838,8 @@ R FunctionMocker<R(Args...)>::InvokeWith(ArgumentTuple&& args)
// Doing so slows down compilation dramatically because the *constructor* of
// std::function<T> is re-instantiated with different template
// parameters each time.
const UninterestingCallCleanupHandler report_uninteresting_call = {
reaction, ss
};
const UninterestingCallCleanupHandler report_uninteresting_call = {reaction,
ss};
return PerformActionAndPrintResult(nullptr, std::move(args), ss.str(), ss);
}
@@ -1890,8 +1889,7 @@ R FunctionMocker<R(Args...)>::InvokeWith(ArgumentTuple&& args)
// std::function<T> is re-instantiated with different template
// parameters each time.
const FailureCleanupHandler handle_failures = {
ss, why, loc, untyped_expectation, found, is_excessive
};
ss, why, loc, untyped_expectation, found, is_excessive};
return PerformActionAndPrintResult(untyped_action, std::move(args), ss.str(),
ss);

View File

@@ -42,6 +42,7 @@
#include <assert.h>
#include <stdlib.h>
#include <cstdint>
#include <iostream>

View File

@@ -53,12 +53,12 @@ class BetweenCardinalityImpl : public CardinalityInterface {
: min_(min >= 0 ? min : 0), max_(max >= min_ ? max : min_) {
std::stringstream ss;
if (min < 0) {
ss << "The invocation lower bound must be >= 0, "
<< "but is actually " << min << ".";
ss << "The invocation lower bound must be >= 0, " << "but is actually "
<< min << ".";
internal::Expect(false, __FILE__, __LINE__, ss.str());
} else if (max < 0) {
ss << "The invocation upper bound must be >= 0, "
<< "but is actually " << max << ".";
ss << "The invocation upper bound must be >= 0, " << "but is actually "
<< max << ".";
internal::Expect(false, __FILE__, __LINE__, ss.str());
} else if (min > max) {
ss << "The invocation upper bound (" << max

View File

@@ -25,7 +25,7 @@ When building GoogleTest as a standalone project, the typical workflow starts
with
```
git clone https://github.com/google/googletest.git -b v1.15.0
git clone https://github.com/google/googletest.git -b v1.16.0
cd googletest # Main directory of the cloned repository.
mkdir build # Create a directory to hold the build output.
cd build

View File

@@ -67,10 +67,10 @@ namespace testing {
// To implement a matcher Foo for type T, define:
// 1. a class FooMatcherMatcher that implements the matcher interface:
// using is_gtest_matcher = void;
// bool MatchAndExplain(const T&, std::ostream*);
// bool MatchAndExplain(const T&, std::ostream*) const;
// (MatchResultListener* can also be used instead of std::ostream*)
// void DescribeTo(std::ostream*);
// void DescribeNegationTo(std::ostream*);
// void DescribeTo(std::ostream*) const;
// void DescribeNegationTo(std::ostream*) const;
//
// 2. a factory function that creates a Matcher<T> object from a
// FooMatcherMatcher.

View File

@@ -126,6 +126,10 @@
#include <span> // NOLINT
#endif // GTEST_INTERNAL_HAS_STD_SPAN
#if GTEST_INTERNAL_HAS_COMPARE_LIB
#include <compare> // NOLINT
#endif // GTEST_INTERNAL_HAS_COMPARE_LIB
namespace testing {
// Definitions in the internal* namespaces are subject to change without notice.
@@ -782,6 +786,41 @@ void PrintTo(const std::shared_ptr<T>& ptr, std::ostream* os) {
(PrintSmartPointer<T>)(ptr, os, 0);
}
#if GTEST_INTERNAL_HAS_COMPARE_LIB
template <typename T>
void PrintOrderingHelper(T ordering, std::ostream* os) {
if (ordering == T::less) {
*os << "(less)";
} else if (ordering == T::greater) {
*os << "(greater)";
} else if (ordering == T::equivalent) {
*os << "(equivalent)";
} else {
*os << "(unknown ordering)";
}
}
inline void PrintTo(std::strong_ordering ordering, std::ostream* os) {
if (ordering == std::strong_ordering::equal) {
*os << "(equal)";
} else {
PrintOrderingHelper(ordering, os);
}
}
inline void PrintTo(std::partial_ordering ordering, std::ostream* os) {
if (ordering == std::partial_ordering::unordered) {
*os << "(unordered)";
} else {
PrintOrderingHelper(ordering, os);
}
}
inline void PrintTo(std::weak_ordering ordering, std::ostream* os) {
PrintOrderingHelper(ordering, os);
}
#endif
// Helper function for printing a tuple. T must be instantiated with
// a tuple type.
template <typename T>

View File

@@ -2533,4 +2533,12 @@ using Variant = ::std::variant<T...>;
#define GTEST_INTERNAL_NEED_REDUNDANT_CONSTEXPR_DECL 1
#endif
#if (defined(__cpp_lib_three_way_comparison) || \
(GTEST_INTERNAL_HAS_INCLUDE(<compare>) && \
GTEST_INTERNAL_CPLUSPLUS_LANG >= 201907L))
#define GTEST_INTERNAL_HAS_COMPARE_LIB 1
#else
#define GTEST_INTERNAL_HAS_COMPARE_LIB 0
#endif
#endif // GOOGLETEST_INCLUDE_GTEST_INTERNAL_GTEST_PORT_H_

View File

@@ -1660,10 +1660,25 @@ std::string GetBoolAssertionFailureMessage(
return msg.GetString();
}
// Helper function for implementing ASSERT_NEAR.
// Helper function for implementing ASSERT_NEAR. Treats infinity as a specific
// value, such that comparing infinity to infinity is equal, the distance
// between -infinity and +infinity is infinity, and infinity <= infinity is
// true.
AssertionResult DoubleNearPredFormat(const char* expr1, const char* expr2,
const char* abs_error_expr, double val1,
double val2, double abs_error) {
// We want to return success when the two values are infinity and at least
// one of the following is true:
// * The values are the same-signed infinity.
// * The error limit itself is infinity.
// This is done here so that we don't end up with a NaN when calculating the
// difference in values.
if (std::isinf(val1) && std::isinf(val2) &&
(std::signbit(val1) == std::signbit(val2) ||
(abs_error > 0.0 && std::isinf(abs_error)))) {
return AssertionSuccess();
}
const double diff = fabs(val1 - val2);
if (diff <= abs_error) return AssertionSuccess();
@@ -3974,6 +3989,12 @@ class XmlUnitTestResultPrinter : public EmptyTestEventListener {
static void OutputXmlTestSuiteForTestResult(::std::ostream* stream,
const TestResult& result);
// Streams a test case XML stanza containing the given test result.
//
// Requires: result.Failed()
static void OutputXmlTestCaseForTestResult(::std::ostream* stream,
const TestResult& result);
// Streams an XML representation of a TestResult object.
static void OutputXmlTestResult(::std::ostream* stream,
const TestResult& result);
@@ -3991,16 +4012,11 @@ class XmlUnitTestResultPrinter : public EmptyTestEventListener {
static void PrintXmlUnitTest(::std::ostream* stream,
const UnitTest& unit_test);
// Produces a string representing the test properties in a result as space
// delimited XML attributes based on the property key="value" pairs.
// When the std::string is not empty, it includes a space at the beginning,
// to delimit this attribute from prior attributes.
static std::string TestPropertiesAsXmlAttributes(const TestResult& result);
// Streams an XML representation of the test properties of a TestResult
// object.
static void OutputXmlTestProperties(std::ostream* stream,
const TestResult& result);
const TestResult& result,
const std::string& indent);
// The output file.
const std::string output_file_;
@@ -4221,6 +4237,15 @@ void XmlUnitTestResultPrinter::OutputXmlTestSuiteForTestResult(
FormatEpochTimeInMillisAsIso8601(result.start_timestamp()));
*stream << ">";
OutputXmlTestCaseForTestResult(stream, result);
// Complete the test suite.
*stream << " </testsuite>\n";
}
// Streams a test case XML stanza containing the given test result.
void XmlUnitTestResultPrinter::OutputXmlTestCaseForTestResult(
::std::ostream* stream, const TestResult& result) {
// Output the boilerplate for a minimal test case with a single test.
*stream << " <testcase";
OutputXmlAttribute(stream, "testcase", "name", "");
@@ -4235,9 +4260,6 @@ void XmlUnitTestResultPrinter::OutputXmlTestSuiteForTestResult(
// Output the actual test result.
OutputXmlTestResult(stream, result);
// Complete the test suite.
*stream << " </testsuite>\n";
}
// Prints an XML representation of a TestInfo object.
@@ -4328,7 +4350,7 @@ void XmlUnitTestResultPrinter::OutputXmlTestResult(::std::ostream* stream,
if (failures == 0 && skips == 0) {
*stream << ">\n";
}
OutputXmlTestProperties(stream, result);
OutputXmlTestProperties(stream, result, /*indent=*/" ");
*stream << " </testcase>\n";
}
}
@@ -4357,13 +4379,18 @@ void XmlUnitTestResultPrinter::PrintXmlTestSuite(std::ostream* stream,
OutputXmlAttribute(
stream, kTestsuite, "timestamp",
FormatEpochTimeInMillisAsIso8601(test_suite.start_timestamp()));
*stream << TestPropertiesAsXmlAttributes(test_suite.ad_hoc_test_result());
}
*stream << ">\n";
OutputXmlTestProperties(stream, test_suite.ad_hoc_test_result(),
/*indent=*/" ");
for (int i = 0; i < test_suite.total_test_count(); ++i) {
if (test_suite.GetTestInfo(i)->is_reportable())
OutputXmlTestInfo(stream, test_suite.name(), *test_suite.GetTestInfo(i));
}
if (test_suite.ad_hoc_test_result().Failed()) {
OutputXmlTestCaseForTestResult(stream, test_suite.ad_hoc_test_result());
}
*stream << " </" << kTestsuite << ">\n";
}
@@ -4393,11 +4420,12 @@ void XmlUnitTestResultPrinter::PrintXmlUnitTest(std::ostream* stream,
OutputXmlAttribute(stream, kTestsuites, "random_seed",
StreamableToString(unit_test.random_seed()));
}
*stream << TestPropertiesAsXmlAttributes(unit_test.ad_hoc_test_result());
OutputXmlAttribute(stream, kTestsuites, "name", "AllTests");
*stream << ">\n";
OutputXmlTestProperties(stream, unit_test.ad_hoc_test_result(),
/*indent=*/" ");
for (int i = 0; i < unit_test.total_test_suite_count(); ++i) {
if (unit_test.GetTestSuite(i)->reportable_test_count() > 0)
PrintXmlTestSuite(stream, *unit_test.GetTestSuite(i));
@@ -4434,21 +4462,8 @@ void XmlUnitTestResultPrinter::PrintXmlTestsList(
*stream << "</" << kTestsuites << ">\n";
}
// Produces a string representing the test properties in a result as space
// delimited XML attributes based on the property key="value" pairs.
std::string XmlUnitTestResultPrinter::TestPropertiesAsXmlAttributes(
const TestResult& result) {
Message attributes;
for (int i = 0; i < result.test_property_count(); ++i) {
const TestProperty& property = result.GetTestProperty(i);
attributes << " " << property.key() << "=" << "\""
<< EscapeXmlAttribute(property.value()) << "\"";
}
return attributes.GetString();
}
void XmlUnitTestResultPrinter::OutputXmlTestProperties(
std::ostream* stream, const TestResult& result) {
std::ostream* stream, const TestResult& result, const std::string& indent) {
const std::string kProperties = "properties";
const std::string kProperty = "property";
@@ -4456,15 +4471,15 @@ void XmlUnitTestResultPrinter::OutputXmlTestProperties(
return;
}
*stream << " <" << kProperties << ">\n";
*stream << indent << "<" << kProperties << ">\n";
for (int i = 0; i < result.test_property_count(); ++i) {
const TestProperty& property = result.GetTestProperty(i);
*stream << " <" << kProperty;
*stream << indent << " <" << kProperty;
*stream << " name=\"" << EscapeXmlAttribute(property.key()) << "\"";
*stream << " value=\"" << EscapeXmlAttribute(property.value()) << "\"";
*stream << "/>\n";
}
*stream << " </" << kProperties << ">\n";
*stream << indent << "</" << kProperties << ">\n";
}
// End XmlUnitTestResultPrinter
@@ -4503,6 +4518,12 @@ class JsonUnitTestResultPrinter : public EmptyTestEventListener {
static void OutputJsonTestSuiteForTestResult(::std::ostream* stream,
const TestResult& result);
// Streams a test case JSON stanza containing the given test result.
//
// Requires: result.Failed()
static void OutputJsonTestCaseForTestResult(::std::ostream* stream,
const TestResult& result);
// Streams a JSON representation of a TestResult object.
static void OutputJsonTestResult(::std::ostream* stream,
const TestResult& result);
@@ -4673,6 +4694,15 @@ void JsonUnitTestResultPrinter::OutputJsonTestSuiteForTestResult(
}
*stream << Indent(6) << "\"testsuite\": [\n";
OutputJsonTestCaseForTestResult(stream, result);
// Finish the test suite.
*stream << "\n" << Indent(6) << "]\n" << Indent(4) << "}";
}
// Streams a test case JSON stanza containing the given test result.
void JsonUnitTestResultPrinter::OutputJsonTestCaseForTestResult(
::std::ostream* stream, const TestResult& result) {
// Output the boilerplate for a new test case.
*stream << Indent(8) << "{\n";
OutputJsonKey(stream, "testcase", "name", "", Indent(10));
@@ -4689,9 +4719,6 @@ void JsonUnitTestResultPrinter::OutputJsonTestSuiteForTestResult(
// Output the actual test result.
OutputJsonTestResult(stream, result);
// Finish the test suite.
*stream << "\n" << Indent(6) << "]\n" << Indent(4) << "}";
}
// Prints a JSON representation of a TestInfo object.
@@ -4836,6 +4863,16 @@ void JsonUnitTestResultPrinter::PrintJsonTestSuite(
OutputJsonTestInfo(stream, test_suite.name(), *test_suite.GetTestInfo(i));
}
}
// If there was a failure in the test suite setup or teardown include that in
// the output.
if (test_suite.ad_hoc_test_result().Failed()) {
if (comma) {
*stream << ",\n";
}
OutputJsonTestCaseForTestResult(stream, test_suite.ad_hoc_test_result());
}
*stream << "\n" << kIndent << "]\n" << Indent(4) << "}";
}

View File

@@ -34,15 +34,26 @@ add_library(pcsx2-rapidyaml
include/c4/yml/detail/stack.hpp
include/c4/yml/emit.def.hpp
include/c4/yml/emit.hpp
include/c4/yml/event_handler_stack.hpp
include/c4/yml/event_handler_tree.hpp
include/c4/yml/filter_processor.hpp
include/c4/yml/fwd.hpp
include/c4/yml/export.hpp
include/c4/yml/node.hpp
include/c4/yml/node_type.hpp
include/c4/yml/parse.hpp
include/c4/yml/parse_engine.def.hpp
include/c4/yml/parse_engine.hpp
include/c4/yml/parser_state.hpp
include/c4/yml/reference_resolver.hpp
include/c4/yml/preprocess.hpp
include/c4/yml/std/map.hpp
include/c4/yml/std/std.hpp
include/c4/yml/std/string.hpp
include/c4/yml/std/vector.hpp
include/c4/yml/tag.hpp
include/c4/yml/tree.hpp
include/c4/yml/version.hpp
include/c4/yml/writer.hpp
include/c4/yml/yml.hpp
include/ryml.hpp
@@ -58,6 +69,10 @@ add_library(pcsx2-rapidyaml
src/c4/yml/parse.cpp
src/c4/yml/preprocess.cpp
src/c4/yml/tree.cpp
src/c4/yml/node_type.cpp
src/c4/yml/reference_resolver.cpp
src/c4/yml/tag.cpp
src/c4/yml/version.cpp
)
target_include_directories(pcsx2-rapidyaml PRIVATE

View File

@@ -18,6 +18,8 @@ template<class T> struct is_blob_type<blob_<T>> : std::integral_constant<bool, t
template<class T> struct is_blob_value_type : std::integral_constant<bool, (std::is_fundamental<T>::value || std::is_trivially_copyable<T>::value)> {};
} // namespace
// NOLINTBEGIN(cppcoreguidelines-special-member-functions,hicpp-special-member-functions)
template<class T>
struct blob_
{
@@ -37,23 +39,25 @@ public:
C4_ALWAYS_INLINE blob_& operator=(blob_ && that) noexcept = default;
C4_ALWAYS_INLINE blob_& operator=(blob_ const& that) noexcept = default;
template<class U, class=typename std::enable_if<std::is_const<T>::value && std::is_same<typename std::add_const<U>::type, T>::value, U>::type> C4_ALWAYS_INLINE blob_(blob_<U> const& that) noexcept : buf(that.buf), len(that.len) {}
template<class U, class=typename std::enable_if<std::is_const<T>::value && std::is_same<typename std::add_const<U>::type, T>::value, U>::type> C4_ALWAYS_INLINE blob_(blob_<U> && that) noexcept : buf(that.buf), len(that.len) {}
template<class U, class=typename std::enable_if<std::is_const<T>::value && std::is_same<typename std::add_const<U>::type, T>::value, U>::type> C4_ALWAYS_INLINE blob_& operator=(blob_<U> && that) noexcept { buf = that.buf; len = that.len; }
template<class U, class=typename std::enable_if<std::is_const<T>::value && std::is_same<typename std::add_const<U>::type, T>::value, U>::type> C4_ALWAYS_INLINE blob_& operator=(blob_<U> const& that) noexcept { buf = that.buf; len = that.len; }
template<class U, class=typename std::enable_if<std::is_const<T>::value && std::is_same<typename std::add_const<U>::type, T>::value, U>::type> C4_ALWAYS_INLINE blob_(blob_<U> const& that) noexcept : buf(that.buf), len(that.len) {} // NOLINT
template<class U, class=typename std::enable_if<std::is_const<T>::value && std::is_same<typename std::add_const<U>::type, T>::value, U>::type> C4_ALWAYS_INLINE blob_(blob_<U> && that) noexcept : buf(that.buf), len(that.len) {} // NOLINT
template<class U, class=typename std::enable_if<std::is_const<T>::value && std::is_same<typename std::add_const<U>::type, T>::value, U>::type> C4_ALWAYS_INLINE blob_& operator=(blob_<U> && that) noexcept { buf = that.buf; len = that.len; } // NOLINT
template<class U, class=typename std::enable_if<std::is_const<T>::value && std::is_same<typename std::add_const<U>::type, T>::value, U>::type> C4_ALWAYS_INLINE blob_& operator=(blob_<U> const& that) noexcept { buf = that.buf; len = that.len; } // NOLINT
C4_ALWAYS_INLINE blob_(void *ptr, size_t n) noexcept : buf(reinterpret_cast<T*>(ptr)), len(n) {}
C4_ALWAYS_INLINE blob_(void const *ptr, size_t n) noexcept : buf(reinterpret_cast<T*>(ptr)), len(n) {}
C4_ALWAYS_INLINE blob_(void *ptr, size_t n) noexcept : buf(reinterpret_cast<T*>(ptr)), len(n) {} // NOLINT
C4_ALWAYS_INLINE blob_(void const *ptr, size_t n) noexcept : buf(reinterpret_cast<T*>(ptr)), len(n) {} // NOLINT
#define _C4_REQUIRE_BLOBTYPE(ty) class=typename std::enable_if<((!detail::is_blob_type<ty>::value) && (detail::is_blob_value_type<ty>::value)), T>::type
template<class U, _C4_REQUIRE_BLOBTYPE(U)> C4_ALWAYS_INLINE blob_(U &var) noexcept : buf(reinterpret_cast<T*>(&var)), len(sizeof(U)) {}
template<class U, _C4_REQUIRE_BLOBTYPE(U)> C4_ALWAYS_INLINE blob_(U *ptr, size_t n) noexcept : buf(reinterpret_cast<T*>(ptr)), len(sizeof(U) * n) { C4_ASSERT(is_aligned(ptr)); }
template<class U, _C4_REQUIRE_BLOBTYPE(U)> C4_ALWAYS_INLINE blob_& operator= (U &var) noexcept { buf = reinterpret_cast<T*>(&var); len = sizeof(U); return *this; }
template<class U, size_t N, _C4_REQUIRE_BLOBTYPE(U)> C4_ALWAYS_INLINE blob_(U (&arr)[N]) noexcept : buf(reinterpret_cast<T*>(arr)), len(sizeof(U) * N) {}
template<class U, size_t N, _C4_REQUIRE_BLOBTYPE(U)> C4_ALWAYS_INLINE blob_& operator= (U (&arr)[N]) noexcept { buf = reinterpret_cast<T*>(arr); len = sizeof(U) * N; return *this; }
template<class U, _C4_REQUIRE_BLOBTYPE(U)> C4_ALWAYS_INLINE blob_(U &var) noexcept : buf(reinterpret_cast<T*>(&var)), len(sizeof(U)) {} // NOLINT
template<class U, _C4_REQUIRE_BLOBTYPE(U)> C4_ALWAYS_INLINE blob_(U *ptr, size_t n) noexcept : buf(reinterpret_cast<T*>(ptr)), len(sizeof(U) * n) { C4_ASSERT(is_aligned(ptr)); } // NOLINT
template<class U, _C4_REQUIRE_BLOBTYPE(U)> C4_ALWAYS_INLINE blob_& operator= (U &var) noexcept { buf = reinterpret_cast<T*>(&var); len = sizeof(U); return *this; } // NOLINT
template<class U, size_t N, _C4_REQUIRE_BLOBTYPE(U)> C4_ALWAYS_INLINE blob_(U (&arr)[N]) noexcept : buf(reinterpret_cast<T*>(arr)), len(sizeof(U) * N) {} // NOLINT
template<class U, size_t N, _C4_REQUIRE_BLOBTYPE(U)> C4_ALWAYS_INLINE blob_& operator= (U (&arr)[N]) noexcept { buf = reinterpret_cast<T*>(arr); len = sizeof(U) * N; return *this; } // NOLINT
#undef _C4_REQUIRE_BLOBTYPE
};
// NOLINTEND(cppcoreguidelines-special-member-functions,hicpp-special-member-functions)
/** an immutable binary blob */
using cblob = blob_<cbyte>;
/** a mutable binary blob */

View File

@@ -55,14 +55,7 @@
# define C4CORE_HAVE_FAST_FLOAT 1
# endif
# if C4CORE_HAVE_FAST_FLOAT
C4_SUPPRESS_WARNING_GCC_WITH_PUSH("-Wsign-conversion")
C4_SUPPRESS_WARNING_GCC("-Warray-bounds")
# if defined(__GNUC__) && __GNUC__ >= 5
C4_SUPPRESS_WARNING_GCC("-Wshift-count-overflow")
# endif
//# include "c4/ext/fast_float.hpp"
#include "fast_float/fast_float.h"
C4_SUPPRESS_WARNING_GCC_POP
# endif
#elif (C4_CPP >= 17)
# define C4CORE_HAVE_FAST_FLOAT 0
@@ -102,15 +95,13 @@
#endif
#if defined(_MSC_VER)
#if defined(_MSC_VER) && !defined(__clang__)
# pragma warning(push)
# pragma warning(disable: 4996) // snprintf/scanf: this function or variable may be unsafe
# if C4_MSVC_VERSION != C4_MSVC_VERSION_2017
# pragma warning(disable: 4800) //'int': forcing value to bool 'true' or 'false' (performance warning)
# endif
#endif
#if defined(__clang__)
#elif defined(__clang__)
# pragma clang diagnostic push
# pragma clang diagnostic ignored "-Wtautological-constant-out-of-range-compare"
# pragma clang diagnostic ignored "-Wformat-nonliteral"
@@ -136,6 +127,7 @@
#define C4_NO_UBSAN_IOVRFLW
#endif
// NOLINTBEGIN(hicpp-signed-bitwise)
namespace c4 {
@@ -144,11 +136,11 @@ namespace c4 {
* Lightweight, very fast generic type-safe wrappers for converting
* individual values to/from strings. These are the main generic
* functions:
* - @ref doc_to_chars and its alias @ref doc_xtoa: implemented by calling @ref itoa()/@ref utoa()/@ref ftoa()/@ref dtoa() (or generically @ref xtoa())
* - @ref doc_from_chars and its alias @ref doc_atox: implemented by calling @ref atoi()/@ref atou()/@ref atof()/@ref atod() (or generically @ref atox())
* - @ref doc_to_chars and its alias @ref xtoa(): implemented by calling @ref itoa() / @ref utoa() / @ref ftoa() / @ref dtoa() (or generically @ref xtoa())
* - @ref doc_from_chars and its alias @ref atox(): implemented by calling @ref atoi() / @ref atou() / @ref atof() / @ref atod() (or generically @ref atox())
* - @ref to_chars_sub()
* - @ref from_chars_first()
* - @ref xtoa()/@ref atox() are implemented in terms @ref write_dec()/@ref read_dec() et al (see @ref doc_write/@ref doc_read())
* - @ref xtoa() and @ref atox() are implemented in terms of @ref write_dec() / @ref read_dec() et al (see @ref doc_write / @ref doc_read())
*
* And also some modest brag is in order: these functions are really
* fast: faster even than C++17 `std::to_chars()` and
@@ -241,7 +233,7 @@ struct is_fixed_length
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
#ifdef _MSC_VER
#if defined(_MSC_VER) && !defined(__clang__)
# pragma warning(push)
#elif defined(__clang__)
# pragma clang diagnostic push
@@ -377,7 +369,7 @@ template<> struct charconv_digits_<4u, false> // uint32_t
static constexpr csubstr max_value_dec() noexcept { return csubstr("4294967295"); }
static constexpr bool is_oct_overflow(csubstr str) noexcept { return !((str.len < 11) || (str.len == 11 && str[0] <= '3')); }
};
template<> struct charconv_digits_<8u, true> // int32_t
template<> struct charconv_digits_<8u, true> // int64_t
{
enum : size_t {
maxdigits_bin = 1 + 2 + 64, // len=67: -9223372036854775808 -0b1000000000000000000000000000000000000000000000000000000000000000
@@ -396,7 +388,7 @@ template<> struct charconv_digits_<8u, true> // int32_t
static constexpr csubstr max_value_dec() noexcept { return csubstr("9223372036854775807"); }
static constexpr bool is_oct_overflow(csubstr str) noexcept { return !((str.len < 22)); }
};
template<> struct charconv_digits_<8u, false>
template<> struct charconv_digits_<8u, false> // uint64_t
{
enum : size_t {
maxdigits_bin = 2 + 64, // len=66: 18446744073709551615 0b1111111111111111111111111111111111111111111111111111111111111111
@@ -545,24 +537,25 @@ C4_CONSTEXPR14 C4_ALWAYS_INLINE unsigned digits_oct(T v_) noexcept
// TODO: is there a better way?
C4_STATIC_ASSERT(std::is_integral<T>::value);
C4_ASSERT(v_ >= 0);
using U = typename
std::conditional<sizeof(T) <= sizeof(unsigned),
unsigned,
typename std::make_unsigned<T>::type>::type;
U v = (U) v_; // safe because we require v_ >= 0
unsigned __n = 1;
const unsigned __b2 = 64u;
const unsigned __b3 = __b2 * 8u;
const unsigned long __b4 = __b3 * 8u;
using U = typename std::conditional<sizeof(T) <= sizeof(unsigned),
unsigned,
typename std::make_unsigned<T>::type>::type;
U v = (U) v_; // safe because we require v_ >= 0 // NOLINT
uint32_t __n = 1;
enum : U {
__b2 = 64u,
__b3 = 64u * 8u,
__b4 = 64u * 8u * 8u,
};
while(true)
{
if(v < 8u)
return __n;
if(v < __b2)
else if(v < __b2)
return __n + 1;
if(v < __b3)
else if(v < __b3)
return __n + 2;
if(v < __b4)
else if(v < __b4)
return __n + 3;
v /= (U) __b4;
__n += 4;
@@ -617,7 +610,7 @@ void write_dec_unchecked(substr buf, T v, unsigned digits_v) noexcept
{
T quo = v;
quo /= T(100);
const auto num = (v - quo * T(100)) << 1u;
const auto num = (v - quo * T(100)) << 1u; // NOLINT
v = quo;
buf.str[--digits_v] = detail::digits0099[num + 1];
buf.str[--digits_v] = detail::digits0099[num];
@@ -782,15 +775,18 @@ template<class T, NumberWriter<T> writer>
size_t write_num_digits(substr buf, T v, size_t num_digits) noexcept
{
C4_STATIC_ASSERT(std::is_integral<T>::value);
size_t ret = writer(buf, v);
const size_t ret = writer(buf, v);
if(ret >= num_digits)
return ret;
else if(ret >= buf.len || num_digits > buf.len)
return num_digits;
C4_ASSERT(num_digits >= ret);
size_t delta = static_cast<size_t>(num_digits - ret);
memmove(buf.str + delta, buf.str, ret);
memset(buf.str, '0', delta);
const size_t delta = static_cast<size_t>(num_digits - ret); // NOLINT
C4_ASSERT(ret + delta <= buf.len);
if(ret)
memmove(buf.str + delta, buf.str, ret);
if(delta)
memset(buf.str, '0', delta);
return num_digits;
}
} // namespace detail
@@ -985,7 +981,9 @@ C4_SUPPRESS_WARNING_GCC_WITH_PUSH("-Wswitch-default")
namespace detail {
inline size_t _itoa2buf(substr buf, size_t pos, csubstr val) noexcept
{
C4_ASSERT(pos < buf.len);
C4_ASSERT(pos + val.len <= buf.len);
C4_ASSERT(val.len > 0);
memcpy(buf.str + pos, val.str, val.len);
return pos + val.len;
}
@@ -1013,7 +1011,7 @@ C4_NO_INLINE size_t _itoa2buf(substr buf, I radix) noexcept
size_t pos = 0;
if(C4_LIKELY(buf.len > 0))
buf.str[pos++] = '-';
switch(radix)
switch(radix) // NOLINT(hicpp-multiway-paths-covered)
{
case I(10):
if(C4_UNLIKELY(buf.len < digits_type::maxdigits_dec))
@@ -1052,7 +1050,7 @@ C4_NO_INLINE size_t _itoa2buf(substr buf, I radix, size_t num_digits) noexcept
size_t needed_digits = 0;
if(C4_LIKELY(buf.len > 0))
buf.str[pos++] = '-';
switch(radix)
switch(radix) // NOLINT(hicpp-multiway-paths-covered)
{
case I(10):
// add 1 to account for -
@@ -1117,7 +1115,7 @@ C4_ALWAYS_INLINE size_t itoa(substr buf, T v) noexcept
}
// when T is the min value (eg i8: -128), negating it
// will overflow, so treat the min as a special case
else if(C4_LIKELY(v != std::numeric_limits<T>::min()))
if(C4_LIKELY(v != std::numeric_limits<T>::min()))
{
v = -v;
unsigned digits = digits_dec(v);
@@ -1160,7 +1158,7 @@ C4_ALWAYS_INLINE size_t itoa(substr buf, T v, T radix) noexcept
++pos;
}
unsigned digits = 0;
switch(radix)
switch(radix) // NOLINT(hicpp-multiway-paths-covered)
{
case T(10):
digits = digits_dec(v);
@@ -1237,7 +1235,7 @@ C4_ALWAYS_INLINE size_t itoa(substr buf, T v, T radix, size_t num_digits) noexce
++pos;
}
unsigned total_digits = 0;
switch(radix)
switch(radix) // NOLINT(hicpp-multiway-paths-covered)
{
case T(10):
total_digits = digits_dec(v);
@@ -1322,7 +1320,7 @@ C4_ALWAYS_INLINE size_t utoa(substr buf, T v, T radix) noexcept
C4_STATIC_ASSERT(std::is_unsigned<T>::value);
C4_ASSERT(radix == 10 || radix == 16 || radix == 2 || radix == 8);
unsigned digits = 0;
switch(radix)
switch(radix) // NOLINT(hicpp-multiway-paths-covered)
{
case T(10):
digits = digits_dec(v);
@@ -1377,7 +1375,7 @@ C4_ALWAYS_INLINE size_t utoa(substr buf, T v, T radix, size_t num_digits) noexce
C4_STATIC_ASSERT(std::is_unsigned<T>::value);
C4_ASSERT(radix == 10 || radix == 16 || radix == 2 || radix == 8);
unsigned total_digits = 0;
switch(radix)
switch(radix) // NOLINT(hicpp-multiway-paths-covered)
{
case T(10):
total_digits = digits_dec(v);
@@ -1601,7 +1599,7 @@ C4_ALWAYS_INLINE size_t atou_first(csubstr str, T *v)
/** @} */
#ifdef _MSC_VER
#if defined(_MSC_VER) && !defined(__clang__)
# pragma warning(pop)
#elif defined(__clang__)
# pragma clang diagnostic pop
@@ -1618,19 +1616,16 @@ C4_ALWAYS_INLINE size_t atou_first(csubstr str, T *v)
namespace detail {
inline bool check_overflow(csubstr str, csubstr limit) noexcept
{
if(str.len == limit.len)
{
for(size_t i = 0; i < limit.len; ++i)
{
if(str[i] < limit[i])
return false;
else if(str[i] > limit[i])
return true;
}
return false;
}
else
if(str.len != limit.len)
return str.len > limit.len;
for(size_t i = 0; i < limit.len; ++i)
{
if(str[i] < limit[i])
return false;
else if(str[i] > limit[i])
return true;
}
return false;
}
} // namespace detail
/** @endcond */
@@ -1714,7 +1709,7 @@ auto overflows(csubstr str) noexcept
* @see doc_overflow_checked for format specifiers to enforce no-overflow reads
*/
template<class T>
auto overflows(csubstr str)
auto overflows(csubstr str) noexcept
-> typename std::enable_if<std::is_signed<T>::value, bool>::type
{
C4_STATIC_ASSERT(std::is_integral<T>::value);
@@ -1762,7 +1757,9 @@ auto overflows(csubstr str)
}
}
else
{
return detail::check_overflow(str.sub(1), detail::charconv_digits<T>::min_value_dec());
}
}
else if(str.str[0] == '0')
{
@@ -1805,7 +1802,9 @@ auto overflows(csubstr str)
}
}
else
{
return detail::check_overflow(str, detail::charconv_digits<T>::max_value_dec());
}
}
/** @} */
@@ -1989,7 +1988,7 @@ C4_ALWAYS_INLINE bool scan_rhex(csubstr s, T *C4_RESTRICT val) noexcept
else if(c == 'p' || c == 'P')
{
++pos;
goto power; // no mantissa given, jump to power
goto power; // no mantissa given, jump to power // NOLINT
}
else
{
@@ -1999,7 +1998,7 @@ C4_ALWAYS_INLINE bool scan_rhex(csubstr s, T *C4_RESTRICT val) noexcept
// mantissa
{
// 0.0625 == 1/16 == value of first digit after the comma
for(T digit = T(0.0625); pos < s.len; ++pos, digit /= T(16))
for(T digit = T(0.0625); pos < s.len; ++pos, digit /= T(16)) // NOLINT
{
const char c = s.str[pos];
if(c >= '0' && c <= '9')
@@ -2011,7 +2010,7 @@ C4_ALWAYS_INLINE bool scan_rhex(csubstr s, T *C4_RESTRICT val) noexcept
else if(c == 'p' || c == 'P')
{
++pos;
goto power; // mantissa finished, jump to power
goto power; // mantissa finished, jump to power // NOLINT
}
else
{
@@ -2172,6 +2171,7 @@ inline size_t atof_first(csubstr str, float * C4_RESTRICT v) noexcept
*/
C4_ALWAYS_INLINE bool atod(csubstr str, double * C4_RESTRICT v) noexcept
{
C4_ASSERT(str.len > 0);
C4_ASSERT(str.triml(" \r\t\n").len == str.len);
#if C4CORE_HAVE_FAST_FLOAT
// fastfloat cannot parse hexadecimal floats
@@ -2228,8 +2228,8 @@ inline size_t atod_first(csubstr str, double * C4_RESTRICT v) noexcept
/** @cond dev */
// on some platforms, (unsigned) int and (unsigned) long
// are not any of the fixed length types above
#define _C4_IF_NOT_FIXED_LENGTH_I(T, ty) C4_ALWAYS_INLINE typename std::enable_if<std:: is_signed<T>::value && !is_fixed_length<T>::value_i, ty>
#define _C4_IF_NOT_FIXED_LENGTH_U(T, ty) C4_ALWAYS_INLINE typename std::enable_if<std::is_unsigned<T>::value && !is_fixed_length<T>::value_u, ty>
#define _C4_IF_NOT_FIXED_LENGTH_I(T, ty) typename std::enable_if<std:: is_signed<T>::value && !is_fixed_length<T>::value_i, ty>
#define _C4_IF_NOT_FIXED_LENGTH_U(T, ty) typename std::enable_if<std::is_unsigned<T>::value && !is_fixed_length<T>::value_u, ty>
/** @endcond*/
@@ -2271,8 +2271,8 @@ C4_ALWAYS_INLINE size_t xtoa(substr s, int64_t v, int64_t radix, size_t num_di
C4_ALWAYS_INLINE size_t xtoa(substr s, float v, int precision, RealFormat_e formatting=FTOA_FLEX) noexcept { return ftoa(s, v, precision, formatting); }
C4_ALWAYS_INLINE size_t xtoa(substr s, double v, int precision, RealFormat_e formatting=FTOA_FLEX) noexcept { return dtoa(s, v, precision, formatting); }
template <class T> _C4_IF_NOT_FIXED_LENGTH_I(T, size_t)::type xtoa(substr buf, T v) noexcept { return itoa(buf, v); }
template <class T> _C4_IF_NOT_FIXED_LENGTH_U(T, size_t)::type xtoa(substr buf, T v) noexcept { return write_dec(buf, v); }
template <class T> C4_ALWAYS_INLINE auto xtoa(substr buf, T v) noexcept -> _C4_IF_NOT_FIXED_LENGTH_I(T, size_t)::type { return itoa(buf, v); }
template <class T> C4_ALWAYS_INLINE auto xtoa(substr buf, T v) noexcept -> _C4_IF_NOT_FIXED_LENGTH_U(T, size_t)::type { return write_dec(buf, v); }
template <class T>
C4_ALWAYS_INLINE size_t xtoa(substr s, T *v) noexcept { return itoa(s, (intptr_t)v, (intptr_t)16); }
@@ -2296,8 +2296,8 @@ C4_ALWAYS_INLINE bool atox(csubstr s, int64_t *C4_RESTRICT v) noexcept { return
C4_ALWAYS_INLINE bool atox(csubstr s, float *C4_RESTRICT v) noexcept { return atof(s, v); }
C4_ALWAYS_INLINE bool atox(csubstr s, double *C4_RESTRICT v) noexcept { return atod(s, v); }
template <class T> _C4_IF_NOT_FIXED_LENGTH_I(T, bool )::type atox(csubstr buf, T *C4_RESTRICT v) noexcept { return atoi(buf, v); }
template <class T> _C4_IF_NOT_FIXED_LENGTH_U(T, bool )::type atox(csubstr buf, T *C4_RESTRICT v) noexcept { return atou(buf, v); }
template <class T> C4_ALWAYS_INLINE auto atox(csubstr buf, T *C4_RESTRICT v) noexcept -> _C4_IF_NOT_FIXED_LENGTH_I(T, bool)::type { return atoi(buf, v); }
template <class T> C4_ALWAYS_INLINE auto atox(csubstr buf, T *C4_RESTRICT v) noexcept -> _C4_IF_NOT_FIXED_LENGTH_U(T, bool)::type { return atou(buf, v); }
template <class T>
C4_ALWAYS_INLINE bool atox(csubstr s, T **v) noexcept { intptr_t tmp; bool ret = atox(s, &tmp); if(ret) { *v = (T*)tmp; } return ret; }
@@ -2336,8 +2336,8 @@ C4_ALWAYS_INLINE size_t to_chars(substr buf, int64_t v) noexcept { return itoa(
C4_ALWAYS_INLINE size_t to_chars(substr buf, float v) noexcept { return ftoa(buf, v); }
C4_ALWAYS_INLINE size_t to_chars(substr buf, double v) noexcept { return dtoa(buf, v); }
template <class T> _C4_IF_NOT_FIXED_LENGTH_I(T, size_t)::type to_chars(substr buf, T v) noexcept { return itoa(buf, v); }
template <class T> _C4_IF_NOT_FIXED_LENGTH_U(T, size_t)::type to_chars(substr buf, T v) noexcept { return write_dec(buf, v); }
template <class T> C4_ALWAYS_INLINE auto to_chars(substr buf, T v) noexcept -> _C4_IF_NOT_FIXED_LENGTH_I(T, size_t)::type { return itoa(buf, v); }
template <class T> C4_ALWAYS_INLINE auto to_chars(substr buf, T v) noexcept -> _C4_IF_NOT_FIXED_LENGTH_U(T, size_t)::type { return write_dec(buf, v); }
template <class T>
C4_ALWAYS_INLINE size_t to_chars(substr s, T *v) noexcept { return itoa(s, (intptr_t)v, (intptr_t)16); }
@@ -2371,8 +2371,8 @@ C4_ALWAYS_INLINE bool from_chars(csubstr buf, int64_t *C4_RESTRICT v) noexcept
C4_ALWAYS_INLINE bool from_chars(csubstr buf, float *C4_RESTRICT v) noexcept { return atof(buf, v); }
C4_ALWAYS_INLINE bool from_chars(csubstr buf, double *C4_RESTRICT v) noexcept { return atod(buf, v); }
template <class T> _C4_IF_NOT_FIXED_LENGTH_I(T, bool )::type from_chars(csubstr buf, T *C4_RESTRICT v) noexcept { return atoi(buf, v); }
template <class T> _C4_IF_NOT_FIXED_LENGTH_U(T, bool )::type from_chars(csubstr buf, T *C4_RESTRICT v) noexcept { return atou(buf, v); }
template <class T> C4_ALWAYS_INLINE auto from_chars(csubstr buf, T *C4_RESTRICT v) noexcept -> _C4_IF_NOT_FIXED_LENGTH_I(T, bool)::type { return atoi(buf, v); }
template <class T> C4_ALWAYS_INLINE auto from_chars(csubstr buf, T *C4_RESTRICT v) noexcept -> _C4_IF_NOT_FIXED_LENGTH_U(T, bool)::type { return atou(buf, v); }
template <class T>
C4_ALWAYS_INLINE bool from_chars(csubstr buf, T **v) noexcept { intptr_t tmp; bool ret = from_chars(buf, &tmp); if(ret) { *v = (T*)tmp; } return ret; }
@@ -2399,8 +2399,8 @@ C4_ALWAYS_INLINE size_t from_chars_first(csubstr buf, int64_t *C4_RESTRICT v) n
C4_ALWAYS_INLINE size_t from_chars_first(csubstr buf, float *C4_RESTRICT v) noexcept { return atof_first(buf, v); }
C4_ALWAYS_INLINE size_t from_chars_first(csubstr buf, double *C4_RESTRICT v) noexcept { return atod_first(buf, v); }
template <class T> _C4_IF_NOT_FIXED_LENGTH_I(T, size_t)::type from_chars_first(csubstr buf, T *C4_RESTRICT v) noexcept { return atoi_first(buf, v); }
template <class T> _C4_IF_NOT_FIXED_LENGTH_U(T, size_t)::type from_chars_first(csubstr buf, T *C4_RESTRICT v) noexcept { return atou_first(buf, v); }
template <class T> C4_ALWAYS_INLINE auto from_chars_first(csubstr buf, T *C4_RESTRICT v) noexcept -> _C4_IF_NOT_FIXED_LENGTH_I(T, size_t)::type { return atoi_first(buf, v); }
template <class T> C4_ALWAYS_INLINE auto from_chars_first(csubstr buf, T *C4_RESTRICT v) noexcept -> _C4_IF_NOT_FIXED_LENGTH_U(T, size_t)::type { return atou_first(buf, v); }
template <class T>
C4_ALWAYS_INLINE size_t from_chars_first(csubstr buf, T **v) noexcept { intptr_t tmp; bool ret = from_chars_first(buf, &tmp); if(ret) { *v = (T*)tmp; } return ret; }
@@ -2657,11 +2657,11 @@ inline size_t to_chars(substr buf, const char * C4_RESTRICT v) noexcept
} // namespace c4
#ifdef _MSC_VER
# pragma warning(pop)
#endif
// NOLINTEND(hicpp-signed-bitwise)
#if defined(__clang__)
#if defined(_MSC_VER) && !defined(__clang__)
# pragma warning(pop)
#elif defined(__clang__)
# pragma clang diagnostic pop
#elif defined(__GNUC__)
# pragma GCC diagnostic pop

View File

@@ -24,7 +24,7 @@
/** @see http://sourceforge.net/p/predef/wiki/Compilers/ for a list of compiler identifier macros */
/** @see https://msdn.microsoft.com/en-us/library/b0084kay.aspx for VS2013 predefined macros */
#if defined(_MSC_VER) && !defined(__clang__)
#if defined(_MSC_VER)
# define C4_MSVC
# define C4_MSVC_VERSION_2022 17
# define C4_MSVC_VERSION_2019 16
@@ -93,6 +93,9 @@
# define C4_CLANG_VERSION __apple_build_version__
# endif
# elif defined(__GNUC__)
# ifdef __MINGW32__
# define C4_MINGW
# endif
# define C4_GCC
# if defined(__GNUC_PATCHLEVEL__)
# define C4_GCC_VERSION C4_VERSION_ENCODED(__GNUC__, __GNUC_MINOR__, __GNUC_PATCHLEVEL__)

View File

@@ -7,7 +7,7 @@
// see also https://sourceforge.net/p/predef/wiki/Architectures/
// see also https://sourceforge.net/p/predef/wiki/Endianness/
// see also https://github.com/googlesamples/android-ndk/blob/android-mk/hello-jni/jni/hello-jni.c
// see http://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/global/qprocessordetection.h
// see also http://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/global/qprocessordetection.h
#ifdef __ORDER_LITTLE_ENDIAN__
# define _C4EL __ORDER_LITTLE_ENDIAN__
@@ -22,7 +22,11 @@
#endif
// mixed byte order (eg, PowerPC or ia64)
#define _C4EM 1111
#define _C4EM 1111 // NOLINT
// NOTE: to find defined macros in a platform,
// g++ <flags> -dM -E - </dev/null | sort
#if defined(__x86_64) || defined(__x86_64__) || defined(__amd64) || defined(_M_X64)
# define C4_CPU_X86_64
@@ -60,11 +64,15 @@
|| defined(__ARM_ARCH_6M__) || defined(__ARM_ARCH_6KZ__) \
|| (defined(__TARGET_ARCH_ARM) && __TARGET_ARCH_ARM >= 6)
# define C4_CPU_ARMV6
# elif defined(__ARM_ARCH_5TEJ__) \
# elif (defined(__ARM_ARCH) && __ARM_ARCH == 5) \
|| defined(__ARM_ARCH_5TEJ__) \
|| defined(__ARM_ARCH_5TE__) \
|| defined(__ARM_ARCH_5T__) \
|| (defined(__TARGET_ARCH_ARM) && __TARGET_ARCH_ARM >= 5)
# define C4_CPU_ARMV5
# elif defined(__ARM_ARCH_4T__) \
# elif (defined(__ARM_ARCH) && __ARM_ARCH == 4) \
|| defined(__ARM_ARCH_4T__) \
|| defined(__ARM_ARCH_4__) \
|| (defined(__TARGET_ARCH_ARM) && __TARGET_ARCH_ARM >= 4)
# define C4_CPU_ARMV4
# else
@@ -145,6 +153,44 @@
# endif
# define C4_BYTE_ORDER _C4EL
#elif defined(__mips__) || defined(_mips) || defined(mips)
# if defined(__mips)
# if __mips == 64
# define C4_CPU_MIPS64
# define C4_WORDSIZE 8
# elif __mips == 32
# define C4_CPU_MIPS32
# define C4_WORDSIZE 4
# endif
# elif defined(__arch64__) || (defined(__SIZE_WIDTH__) && __SIZE_WIDTH__ == 64) || (defined(__LP64__) && __LP64__)
# define C4_CPU_MIPS64
# define C4_WORDSIZE 8
# elif defined(__arch32__) || (defined(__SIZE_WIDTH__) && __SIZE_WIDTH__ == 32) || (defined(__LP32__) && __LP32__)
# define C4_CPU_MIPS32
# define C4_WORDSIZE 4
# else
# error "unknown mips architecture"
# endif
# if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
# define C4_BYTE_ORDER _C4EB
# elif __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
# define C4_BYTE_ORDER _C4EL
# else
# error "unknown mips endianness"
# endif
#elif defined(__sparc__) || defined(__sparc) || defined(sparc)
# if defined(__arch64__) || (defined(__SIZE_WIDTH__) && __SIZE_WIDTH__ == 64) || (defined(__LP64__) && __LP64__)
# define C4_CPU_SPARC64
# define C4_WORDSIZE 8
# elif defined(__arch32__) || (defined(__SIZE_WIDTH__) && __SIZE_WIDTH__ == 32) || (defined(__LP32__) && __LP32__)
# define C4_CPU_SPARC32
# define C4_WORDSIZE 4
# else
# error "unknown sparc architecture"
# endif
# define C4_BYTE_ORDER _C4EB
#elif defined(SWIG)
# error "please define CPU architecture macros when compiling with swig"

File diff suppressed because it is too large Load Diff

View File

@@ -63,6 +63,7 @@ struct fail_type__ {};
#else
# ifdef __clang__
# pragma clang diagnostic push
# pragma clang diagnostic ignored "-Wundef"
# if !defined(__APPLE_CC__)
# if __clang_major__ >= 10
# pragma clang diagnostic ignored "-Wgnu-inline-cpp-without-extern" // debugbreak/debugbreak.h:50:16: error: 'gnu_inline' attribute without 'extern' in C++ treated as externally available, this changed in Clang 10 [-Werror,-Wgnu-inline-cpp-without-extern]
@@ -73,12 +74,15 @@ struct fail_type__ {};
# endif
# endif
# elif defined(__GNUC__)
# pragma GCC diagnostic push
# pragma GCC diagnostic ignored "-Wundef"
# endif
# include <c4/ext/debugbreak/debugbreak.h>
# define C4_DEBUG_BREAK() if(c4::is_debugger_attached()) { ::debug_break(); }
# ifdef __clang__
# pragma clang diagnostic pop
# elif defined(__GNUC__)
# pragma GCC diagnostic pop
# endif
#endif
@@ -114,17 +118,17 @@ namespace c4 {
typedef enum : uint32_t {
/** when an error happens and the debugger is attached, call C4_DEBUG_BREAK().
* Without effect otherwise. */
ON_ERROR_DEBUGBREAK = 0x01 << 0,
ON_ERROR_DEBUGBREAK = 0x01u << 0u,
/** when an error happens log a message. */
ON_ERROR_LOG = 0x01 << 1,
ON_ERROR_LOG = 0x01u << 1u,
/** when an error happens invoke a callback if it was set with
* set_error_callback(). */
ON_ERROR_CALLBACK = 0x01 << 2,
ON_ERROR_CALLBACK = 0x01u << 2u,
/** when an error happens call std::terminate(). */
ON_ERROR_ABORT = 0x01 << 3,
ON_ERROR_ABORT = 0x01u << 3u,
/** when an error happens and exceptions are enabled throw an exception.
* Without effect otherwise. */
ON_ERROR_THROW = 0x01 << 4,
ON_ERROR_THROW = 0x01u << 4u,
/** the default flags. */
ON_ERROR_DEFAULTS = ON_ERROR_DEBUGBREAK|ON_ERROR_LOG|ON_ERROR_CALLBACK|ON_ERROR_ABORT
} ErrorFlags_e;
@@ -140,7 +144,7 @@ C4CORE_EXPORT error_callback_type get_error_callback();
//-----------------------------------------------------------------------------
/** RAII class controling the error settings inside a scope. */
struct ScopedErrorSettings
struct ScopedErrorSettings // NOLINT(cppcoreguidelines-special-member-functions,hicpp-special-member-functions)
{
error_flags m_flags;
error_callback_type m_callback;
@@ -177,7 +181,8 @@ struct ScopedErrorSettings
/** source location */
struct srcloc;
C4CORE_EXPORT void handle_error(srcloc s, const char *fmt, ...);
// watchout: for VS the [[noreturn]] needs to come before other annotations like C4CORE_EXPORT
[[noreturn]] C4CORE_EXPORT void handle_error(srcloc s, const char *fmt, ...);
C4CORE_EXPORT void handle_warning(srcloc s, const char *fmt, ...);

View File

@@ -8,7 +8,7 @@
#include "c4/blob.hpp"
#ifdef _MSC_VER
#if defined(_MSC_VER) && !defined(__clang__)
# pragma warning(push)
# if C4_MSVC_VERSION != C4_MSVC_VERSION_2017
# pragma warning(disable: 4800) // forcing value to bool 'true' or 'false' (performance warning)
@@ -20,6 +20,7 @@
# pragma GCC diagnostic push
# pragma GCC diagnostic ignored "-Wuseless-cast"
#endif
// NOLINTBEGIN(cppcoreguidelines-pro-type-reinterpret-cast,*avoid-goto*)
/** @defgroup doc_format_utils Format utilities
*
@@ -323,7 +324,7 @@ to_chars(substr buf, fmt::integral_padded_<T> fmt)
return utoa(buf, fmt.val, fmt.radix, fmt.num_digits);
}
/** read an format an integer unsigned type
/** read an integer type, detecting overflow (returns false on overflow)
* @ingroup doc_from_chars */
template<class T>
C4_ALWAYS_INLINE bool from_chars(csubstr s, fmt::overflow_checked_<T> wrapper)
@@ -332,6 +333,15 @@ C4_ALWAYS_INLINE bool from_chars(csubstr s, fmt::overflow_checked_<T> wrapper)
return atox(s, wrapper.val);
return false;
}
/** read an integer type, detecting overflow (returns false on overflow)
* @ingroup doc_from_chars */
template<class T>
C4_ALWAYS_INLINE bool from_chars(csubstr s, fmt::overflow_checked_<T> *wrapper)
{
if(C4_LIKELY(!overflows<T>(s)))
return atox(s, wrapper->val);
return false;
}
//-----------------------------------------------------------------------------
@@ -967,7 +977,6 @@ inline CharOwningContainer catseprs(Sep const& C4_RESTRICT sep, Args const& C4_R
* the result. The buffer is appended to.
*
* @return a csubstr of the appended part
* @ingroup formatting_functions
* @ingroup doc_catsep */
template<class CharOwningContainer, class Sep, class... Args>
inline csubstr catseprs_append(CharOwningContainer * C4_RESTRICT cont, Sep const& C4_RESTRICT sep, Args const& C4_RESTRICT ...args)
@@ -1019,7 +1028,6 @@ inline CharOwningContainer formatrs(csubstr fmt, Args const& C4_RESTRICT ...args
* arguments, resizing the container as needed to contain the
* result. The buffer is appended to.
* @return the region newly appended to the original container
* @ingroup formatting_functions
* @ingroup doc_format */
template<class CharOwningContainer, class... Args>
inline csubstr formatrs_append(CharOwningContainer * C4_RESTRICT cont, csubstr fmt, Args const& C4_RESTRICT ...args)
@@ -1038,6 +1046,7 @@ retry:
} // namespace c4
// NOLINTEND(cppcoreguidelines-pro-type-reinterpret-cast,*avoid-goto*)
#ifdef _MSC_VER
# pragma warning(pop)
#elif defined(__clang__)

View File

@@ -333,7 +333,7 @@ namespace detail {
#ifdef __GNUC__
# define C4_DONT_OPTIMIZE(var) c4::detail::dont_optimize(var)
template< class T >
C4_ALWAYS_INLINE void dont_optimize(T const& value) { asm volatile("" : : "g"(value) : "memory"); }
C4_ALWAYS_INLINE void dont_optimize(T const& value) { asm volatile("" : : "g"(value) : "memory"); } // NOLINT
#else
# define C4_DONT_OPTIMIZE(var) c4::detail::use_char_pointer(reinterpret_cast< const char* >(&var))
void use_char_pointer(char const volatile*);

View File

@@ -25,6 +25,8 @@
/** @file memory_util.hpp Some memory utilities. */
// NOLINTBEGIN(google-runtime-int)
namespace c4 {
C4_SUPPRESS_WARNING_GCC_CLANG_WITH_PUSH("-Wold-style-cast")
@@ -696,8 +698,8 @@ struct tight_pair<First, Second, tpc_same_empty> : public First
C4_ALWAYS_INLINE C4_CONSTEXPR14 First & first () { return static_cast<First &>(*this); }
C4_ALWAYS_INLINE C4_CONSTEXPR14 First const& first () const { return static_cast<First const&>(*this); }
C4_ALWAYS_INLINE C4_CONSTEXPR14 Second & second() { return reinterpret_cast<Second &>(*this); }
C4_ALWAYS_INLINE C4_CONSTEXPR14 Second const& second() const { return reinterpret_cast<Second const&>(*this); }
C4_ALWAYS_INLINE C4_CONSTEXPR14 Second & second() { return reinterpret_cast<Second &>(*this); } // NOLINT
C4_ALWAYS_INLINE C4_CONSTEXPR14 Second const& second() const { return reinterpret_cast<Second const&>(*this); } // NOLINT
};
template<class First, class Second>
@@ -775,4 +777,6 @@ C4_SUPPRESS_WARNING_GCC_CLANG_POP
} // namespace c4
// NOLINTEND(google-runtime-int)
#endif /* _C4_MEMORY_UTIL_HPP_ */

View File

@@ -67,12 +67,9 @@ static inline void _do_reverse(C *C4_RESTRICT first, C *C4_RESTRICT last)
* @see a [quickstart
* sample](https://rapidyaml.readthedocs.io/latest/doxygen/group__doc__quickstart.html#ga43e253da0692c13967019446809c1113)
* in rapidyaml's documentation.
*
* @see @ref substr and @ref to_substr()
* @see @ref csubstr and @ref to_csubstr()
*/
template<class C>
struct C4CORE_EXPORT basic_substring
struct C4CORE_EXPORT basic_substring // NOLINT(cppcoreguidelines-special-member-functions,hicpp-special-member-functions)
{
public:
@@ -530,11 +527,11 @@ public:
/** @name Lookup methods */
/** @{ */
inline size_t find(const C c, size_t start_pos=0) const
size_t find(const C c, size_t start_pos=0) const
{
return first_of(c, start_pos);
}
inline size_t find(ro_substr pattern, size_t start_pos=0) const
size_t find(ro_substr pattern, size_t start_pos=0) const
{
C4_ASSERT(start_pos == npos || (start_pos >= 0 && start_pos <= len));
if(len < pattern.len) return npos;
@@ -561,7 +558,7 @@ public:
public:
/** count the number of occurrences of c */
inline size_t count(const C c, size_t pos=0) const
size_t count(const C c, size_t pos=0) const
{
C4_ASSERT(pos >= 0 && pos <= len);
size_t num = 0;
@@ -575,7 +572,7 @@ public:
}
/** count the number of occurrences of s */
inline size_t count(ro_substr c, size_t pos=0) const
size_t count(ro_substr c, size_t pos=0) const
{
C4_ASSERT(pos >= 0 && pos <= len);
size_t num = 0;
@@ -589,14 +586,14 @@ public:
}
/** get the substr consisting of the first occurrence of @p c after @p pos, or an empty substr if none occurs */
inline basic_substring select(const C c, size_t pos=0) const
basic_substring select(const C c, size_t pos=0) const
{
pos = find(c, pos);
return pos != npos ? sub(pos, 1) : basic_substring();
}
/** get the substr consisting of the first occurrence of @p pattern after @p pos, or an empty substr if none occurs */
inline basic_substring select(ro_substr pattern, size_t pos=0) const
basic_substring select(ro_substr pattern, size_t pos=0) const
{
pos = find(pattern, pos);
return pos != npos ? sub(pos, pattern.len) : basic_substring();
@@ -608,7 +605,7 @@ public:
{
size_t which;
size_t pos;
inline operator bool() const { return which != NONE && pos != npos; }
operator bool() const { return which != NONE && pos != npos; }
};
first_of_any_result first_of_any(ro_substr s0, ro_substr s1) const
@@ -1307,12 +1304,12 @@ public:
else if(c == '.')
{
++pos;
goto fractional_part_dec;
goto fractional_part_dec; // NOLINT
}
else if(c == 'e' || c == 'E')
{
++pos;
goto power_part_dec;
goto power_part_dec; // NOLINT
}
else if(_is_delim_char(c))
{
@@ -1341,7 +1338,7 @@ public:
else if(c == 'e' || c == 'E')
{
++pos;
goto power_part_dec;
goto power_part_dec; // NOLINT
}
else if(_is_delim_char(c))
{
@@ -1394,12 +1391,12 @@ public:
else if(c == '.')
{
++pos;
goto fractional_part_hex;
goto fractional_part_hex; // NOLINT
}
else if(c == 'p' || c == 'P')
{
++pos;
goto power_part_hex;
goto power_part_hex; // NOLINT
}
else if(_is_delim_char(c))
{
@@ -1428,7 +1425,7 @@ public:
else if(c == 'p' || c == 'P')
{
++pos;
goto power_part_hex;
goto power_part_hex; // NOLINT
}
else if(_is_delim_char(c))
{
@@ -1484,12 +1481,12 @@ public:
else if(c == '.')
{
++pos;
goto fractional_part_bin;
goto fractional_part_bin; // NOLINT
}
else if(c == 'p' || c == 'P')
{
++pos;
goto power_part_bin;
goto power_part_bin; // NOLINT
}
else if(_is_delim_char(c))
{
@@ -1518,7 +1515,7 @@ public:
else if(c == 'p' || c == 'P')
{
++pos;
goto power_part_bin;
goto power_part_bin; // NOLINT
}
else if(_is_delim_char(c))
{
@@ -1574,12 +1571,12 @@ public:
else if(c == '.')
{
++pos;
goto fractional_part_oct;
goto fractional_part_oct; // NOLINT
}
else if(c == 'p' || c == 'P')
{
++pos;
goto power_part_oct;
goto power_part_oct; // NOLINT
}
else if(_is_delim_char(c))
{
@@ -1608,7 +1605,7 @@ public:
else if(c == 'p' || c == 'P')
{
++pos;
goto power_part_oct;
goto power_part_oct; // NOLINT
}
else if(_is_delim_char(c))
{
@@ -1714,7 +1711,7 @@ private:
}
split_iterator_impl& operator++ () { _tick(); return *this; }
split_iterator_impl operator++ (int) { split_iterator_impl it = *this; _tick(); return it; }
split_iterator_impl operator++ (int) { split_iterator_impl it = *this; _tick(); return it; } // NOLINT
basic_substring& operator* () { return m_str; }
basic_substring* operator-> () { return &m_str; }
@@ -2069,7 +2066,7 @@ public:
C4_REQUIRE_RW(basic_substring) erase_range(size_t first, size_t last)
{
C4_ASSERT(first <= last);
return erase(first, static_cast<size_t>(last-first));
return erase(first, static_cast<size_t>(last-first)); // NOLINT
}
/** erase a part of the string.

View File

@@ -53,7 +53,7 @@ szconv(SizeIn sz) noexcept
template<class SizeOut, class SizeIn>
C4_ALWAYS_INLINE
typename std::enable_if<is_narrower_size<SizeOut, SizeIn>::value, SizeOut>::type
szconv(SizeIn sz) C4_NOEXCEPT_X
szconv(SizeIn sz)
{
C4_XASSERT(sz >= 0);
C4_XASSERT_MSG((SizeIn)sz <= (SizeIn)std::numeric_limits<SizeOut>::max(), "size conversion overflow: in=%zu", (size_t)sz);

View File

@@ -17,6 +17,8 @@
/** @defgroup types Type utilities */
// NOLINTBEGIN(bugprone-macro-parentheses)
namespace c4 {
/** @defgroup intrinsic_types Intrinsic types
@@ -114,17 +116,17 @@ using fastcref = typename std::conditional<c4::cref_uses_val<T>::value, T, T con
//--------------------------------------------------
/** Just what its name says. Useful sometimes as a default empty policy class. */
struct EmptyStruct
struct EmptyStruct // NOLINT
{
template<class... T> EmptyStruct(T && ...){}
template<class... T> EmptyStruct(T && ...){} // NOLINT
};
/** Just what its name says. Useful sometimes as a default policy class to
* be inherited from. */
struct EmptyStructVirtual
struct EmptyStructVirtual // NOLINT
{
virtual ~EmptyStructVirtual() = default;
template<class... T> EmptyStructVirtual(T && ...){}
template<class... T> EmptyStructVirtual(T && ...){} // NOLINT
};
@@ -159,7 +161,7 @@ struct Padded : public T
using T::T;
using T::operator=;
Padded(T const& val) : T(val) {}
Padded(T && val) : T(val) {}
Padded(T && val) : T(std::forward<T>(val)) {} // NOLINT
char ___c4padspace___[BytesToPadAtEnd];
};
#pragma pack(pop)
@@ -170,7 +172,7 @@ struct Padded<T, 0> : public T
using T::T;
using T::operator=;
Padded(T const& val) : T(val) {}
Padded(T && val) : T(val) {}
Padded(T && val) : T(std::forward<T>(val)) {} // NOLINT
};
/** make T have a size which is at least Min bytes */
@@ -500,4 +502,6 @@ using index_sequence_for = make_index_sequence<sizeof...(_Tp)>;
} // namespace c4
// NOLINTEND(bugprone-macro-parentheses)
#endif /* _C4_TYPES_HPP_ */

View File

@@ -6,10 +6,67 @@
#include <stddef.h>
#include <stdint.h>
/** @file utf.hpp utilities for UTF and Byte Order Mark */
namespace c4 {
substr decode_code_point(substr out, csubstr code_point);
size_t decode_code_point(uint8_t *C4_RESTRICT buf, size_t buflen, const uint32_t code);
/** @defgroup doc_utf UTF utilities
* @{ */
/** skip the Byte Order Mark, or get the full string if there is Byte Order Mark.
* @see Implements the Byte Order Marks as described in https://en.wikipedia.org/wiki/Byte_order_mark#Byte-order_marks_by_encoding */
C4CORE_EXPORT substr skip_bom(substr s);
/** skip the Byte Order Mark, or get the full string if there is Byte Order Mark
* @see Implements the Byte Order Marks as described in https://en.wikipedia.org/wiki/Byte_order_mark#Byte-order_marks_by_encoding */
C4CORE_EXPORT csubstr skip_bom(csubstr s);
/** get the Byte Order Mark, or an empty string if there is no Byte Order Mark
* @see Implements the Byte Order Marks as described in https://en.wikipedia.org/wiki/Byte_order_mark#Byte-order_marks_by_encoding */
C4CORE_EXPORT substr get_bom(substr s);
/** get the Byte Order Mark, or an empty string if there is no Byte Order Mark
* @see Implements the Byte Order Marks as described in https://en.wikipedia.org/wiki/Byte_order_mark#Byte-order_marks_by_encoding */
C4CORE_EXPORT csubstr get_bom(csubstr s);
/** return the position of the first character not belonging to the
* Byte Order Mark, or 0 if there is no Byte Order Mark.
* @see Implements the Byte Order Marks as described in https://en.wikipedia.org/wiki/Byte_order_mark#Byte-order_marks_by_encoding */
C4CORE_EXPORT size_t first_non_bom(csubstr s);
/** decode the given @p code_point, writing into the output string in
* @p out.
*
* @param out the output string. must have at least 4 bytes (this is
* asserted), and must not have a null string.
*
* @param code_point: must have length in ]0,8], and must not begin
* with any of `U+`,`\\x`,`\\u,`\\U`,`0` (asserted)
*
* @return the part of @p out that was written, which will always be
* at most 4 bytes.
*/
C4CORE_EXPORT substr decode_code_point(substr out, csubstr code_point);
/** decode the given @p code point, writing into the output string @p
* buf, of size @p buflen
*
* @param buf the output string. must have at least 4 bytes (this is
* asserted), and must not be null
*
* @param buflen the length of the output string. must be at least 4
*
* @param code: the code point must have length in ]0,8], and must not begin
* with any of `U+`,`\\x`,`\\u,`\\U`,`0` (asserted)
*
* @return the part of @p out that was written, which will always be
* at most 4 bytes.
*/
size_t decode_code_point(uint8_t *C4_RESTRICT buf, size_t buflen, uint32_t code);
/** @} */
} // namespace c4

View File

@@ -5,8 +5,50 @@
#include <cstddef>
#include <c4/substr.hpp>
#include <c4/charconv.hpp>
#include <c4/dump.hpp>
#include <c4/yml/export.hpp>
#if defined(C4_MSVC) || defined(C4_MINGW)
#include <malloc.h>
#else
#include <alloca.h>
#endif
//-----------------------------------------------------------------------------
#ifndef RYML_ERRMSG_SIZE
/// size for the error message buffer
#define RYML_ERRMSG_SIZE (1024)
#endif
#ifndef RYML_LOGBUF_SIZE
/// size for the buffer used to format individual values to string
/// while preparing an error message. This is only used for formatting
/// individual values in the message; final messages will be larger
/// than this value (see @ref RYML_ERRMSG_SIZE). This is also used for
/// the detailed debug log messages when RYML_DBG is defined.
#define RYML_LOGBUF_SIZE (256)
#endif
#ifndef RYML_LOGBUF_SIZE_MAX
/// size for the fallback larger log buffer. When @ref
/// RYML_LOGBUF_SIZE is not large enough to convert a value to string,
/// then temporary stack memory is allocated up to
/// RYML_LOGBUF_SIZE_MAX. This limit is in place to prevent a stack
/// overflow. If the printed value requires more than
/// RYML_LOGBUF_SIZE_MAX, the value is silently skipped.
#define RYML_LOGBUF_SIZE_MAX (1024)
#endif
#ifndef RYML_LOCATIONS_SMALL_THRESHOLD
/// threshold at which a location search will revert from linear to
/// binary search.
#define RYML_LOCATIONS_SMALL_THRESHOLD (30)
#endif
//-----------------------------------------------------------------------------
// Specify groups to have a predefined topic order in doxygen:
@@ -83,6 +125,11 @@
*
*/
/** @defgroup doc_ref_utils Anchor/Reference utilities
*
* @see sample::sample_anchors_and_aliases
* */
/** @defgroup doc_tag_utils Tag utilities
* @see sample::sample_tags
*/
@@ -134,11 +181,13 @@
# define RYML_ASSERT(cond) RYML_CHECK(cond)
# define RYML_ASSERT_MSG(cond, msg) RYML_CHECK_MSG(cond, msg)
# define _RYML_CB_ASSERT(cb, cond) _RYML_CB_CHECK((cb), (cond))
# define _RYML_CB_ASSERT_(cb, cond, loc) _RYML_CB_CHECK((cb), (cond), (loc))
# define RYML_NOEXCEPT
#else
# define RYML_ASSERT(cond)
# define RYML_ASSERT_MSG(cond, msg)
# define _RYML_CB_ASSERT(cb, cond)
# define _RYML_CB_ASSERT_(cb, cond, loc)
# define RYML_NOEXCEPT noexcept
#endif
@@ -148,7 +197,7 @@
do { \
if(C4_UNLIKELY(!(cond))) \
{ \
RYML_DEBUG_BREAK() \
RYML_DEBUG_BREAK(); \
c4::yml::error("check failed: " #cond, c4::yml::Location(__FILE__, __LINE__, 0)); \
C4_UNREACHABLE_AFTER_ERR(); \
} \
@@ -159,7 +208,7 @@
{ \
if(C4_UNLIKELY(!(cond))) \
{ \
RYML_DEBUG_BREAK() \
RYML_DEBUG_BREAK(); \
c4::yml::error(msg ": check failed: " #cond, c4::yml::Location(__FILE__, __LINE__, 0)); \
C4_UNREACHABLE_AFTER_ERR(); \
} \
@@ -167,17 +216,16 @@
#if defined(RYML_DBG) && !defined(NDEBUG) && !defined(C4_NO_DEBUG_BREAK)
# define RYML_DEBUG_BREAK() \
{ \
do { \
if(c4::get_error_flags() & c4::ON_ERROR_DEBUGBREAK) \
{ \
C4_DEBUG_BREAK(); \
} \
}
} while(0)
#else
# define RYML_DEBUG_BREAK()
#endif
/** @endcond */
@@ -190,11 +238,33 @@ namespace yml {
C4_SUPPRESS_WARNING_GCC_CLANG_WITH_PUSH("-Wold-style-cast")
enum : size_t {
/** a null position */
npos = size_t(-1),
#ifndef RYML_ID_TYPE
/** The type of a node id in the YAML tree. In the future, the default
* will likely change to int32_t, which was observed to be faster.
* @see id_type */
#define RYML_ID_TYPE size_t
#endif
/** The type of a node id in the YAML tree; to override the default
* type, define the macro @ref RYML_ID_TYPE to a suitable integer
* type. */
using id_type = RYML_ID_TYPE;
static_assert(std::is_integral<id_type>::value, "id_type must be an integer type");
C4_SUPPRESS_WARNING_GCC_WITH_PUSH("-Wuseless-cast")
enum : id_type {
/** an index to none */
NONE = size_t(-1)
NONE = id_type(-1),
};
C4_SUPPRESS_WARNING_GCC_CLANG_POP
enum : size_t {
/** a null string position */
npos = size_t(-1)
};
@@ -212,28 +282,40 @@ struct RYML_EXPORT LineCol
//! column
size_t col;
LineCol() : offset(), line(), col() {}
LineCol() = default;
//! construct from line and column
LineCol(size_t l, size_t c) : offset(0), line(l), col(c) {}
//! construct from offset, line and column
LineCol(size_t o, size_t l, size_t c) : offset(o), line(l), col(c) {}
};
static_assert(std::is_trivial<LineCol>::value, "LineCol not trivial");
static_assert(std::is_standard_layout<LineCol>::value, "Location not trivial");
//! a source file position
struct RYML_EXPORT Location : public LineCol
struct RYML_EXPORT Location
{
//! number of bytes from the beginning of the source buffer
size_t offset;
//! line
size_t line;
//! column
size_t col;
//! file name
csubstr name;
operator bool () const { return !name.empty() || line != 0 || offset != 0; }
operator bool () const { return !name.empty() || line != 0 || offset != 0 || col != 0; }
operator LineCol const& () const { return reinterpret_cast<LineCol const&>(*this); } // NOLINT
Location() : LineCol(), name() {}
Location( size_t l, size_t c) : LineCol{ l, c}, name( ) {}
Location( csubstr n, size_t l, size_t c) : LineCol{ l, c}, name(n) {}
Location( csubstr n, size_t b, size_t l, size_t c) : LineCol{b, l, c}, name(n) {}
Location(const char *n, size_t l, size_t c) : LineCol{ l, c}, name(to_csubstr(n)) {}
Location(const char *n, size_t b, size_t l, size_t c) : LineCol{b, l, c}, name(to_csubstr(n)) {}
Location() = default;
Location( size_t l, size_t c) : offset( ), line(l), col(c), name( ) {}
Location( size_t b, size_t l, size_t c) : offset(b), line(l), col(c), name( ) {}
Location( csubstr n, size_t l, size_t c) : offset( ), line(l), col(c), name(n) {}
Location( csubstr n, size_t b, size_t l, size_t c) : offset(b), line(l), col(c), name(n) {}
Location(const char *n, size_t l, size_t c) : offset( ), line(l), col(c), name(to_csubstr(n)) {}
Location(const char *n, size_t b, size_t l, size_t c) : offset(b), line(l), col(c), name(to_csubstr(n)) {}
};
static_assert(std::is_standard_layout<Location>::value, "Location not trivial");
//-----------------------------------------------------------------------------
@@ -300,7 +382,7 @@ struct RYML_EXPORT Callbacks
/** Construct an object with the default callbacks. If
* @ref RYML_NO_DEFAULT_CALLBACKS is defined, the object will have null
* members.*/
Callbacks();
Callbacks() noexcept;
/** Construct an object with the given callbacks.
*
@@ -338,6 +420,20 @@ struct RYML_EXPORT Callbacks
/** @} */
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
typedef enum {
NOBOM,
UTF8,
UTF16LE,
UTF16BE,
UTF32LE,
UTF32BE,
} Encoding_e;
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
@@ -364,30 +460,30 @@ template<size_t N>
}
#define _RYML_CB_ERR(cb, msg_literal) \
_RYML_CB_ERR_(cb, msg_literal, c4::yml::Location(__FILE__, 0, __LINE__, 0))
#define _RYML_CB_CHECK(cb, cond) \
_RYML_CB_CHECK_(cb, cond, c4::yml::Location(__FILE__, 0, __LINE__, 0))
#define _RYML_CB_ERR_(cb, msg_literal, loc) \
do \
{ \
const char msg[] = msg_literal; \
RYML_DEBUG_BREAK() \
c4::yml::error((cb), \
msg, sizeof(msg), \
c4::yml::Location(__FILE__, 0, __LINE__, 0)); \
RYML_DEBUG_BREAK(); \
c4::yml::error((cb), msg, sizeof(msg)-1, loc); \
C4_UNREACHABLE_AFTER_ERR(); \
} while(0)
#define _RYML_CB_CHECK(cb, cond) \
#define _RYML_CB_CHECK_(cb, cond, loc) \
do \
{ \
if(!(cond)) \
if(C4_UNLIKELY(!(cond))) \
{ \
const char msg[] = "check failed: " #cond; \
RYML_DEBUG_BREAK() \
c4::yml::error((cb), \
msg, sizeof(msg), \
c4::yml::Location(__FILE__, 0, __LINE__, 0)); \
RYML_DEBUG_BREAK(); \
c4::yml::error((cb), msg, sizeof(msg)-1, loc); \
C4_UNREACHABLE_AFTER_ERR(); \
} \
} while(0)
#define _RYML_CB_ALLOC_HINT(cb, T, num, hint) (T*) (cb).m_allocate((num) * sizeof(T), (hint), (cb).m_user_data)
#define _RYML_CB_ALLOC(cb, T, num) _RYML_CB_ALLOC_HINT((cb), (T), (num), nullptr)
#define _RYML_CB_ALLOC(cb, T, num) _RYML_CB_ALLOC_HINT((cb), T, (num), nullptr)
#define _RYML_CB_FREE(cb, buf, T, num) \
do { \
(cb).m_free((buf), (num) * sizeof(T), (cb).m_user_data); \
@@ -395,11 +491,54 @@ do \
} while(0)
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
typedef enum {
BLOCK_LITERAL, //!< keep newlines (|)
BLOCK_FOLD //!< replace newline with single space (>)
} BlockStyle_e;
typedef enum {
CHOMP_CLIP, //!< single newline at end (default)
CHOMP_STRIP, //!< no newline at end (-)
CHOMP_KEEP //!< all newlines from end (+)
} BlockChomp_e;
/** Abstracts the fact that a scalar filter result may not fit in the
* intended memory. */
struct FilterResult
{
C4_ALWAYS_INLINE bool valid() const noexcept { return str.str != nullptr; }
C4_ALWAYS_INLINE size_t required_len() const noexcept { return str.len; }
C4_ALWAYS_INLINE csubstr get() const { RYML_ASSERT(valid()); return str; }
csubstr str;
};
/** Abstracts the fact that a scalar filter result may not fit in the
* intended memory. */
struct FilterResultExtending
{
C4_ALWAYS_INLINE bool valid() const noexcept { return str.str != nullptr; }
C4_ALWAYS_INLINE size_t required_len() const noexcept { return reqlen; }
C4_ALWAYS_INLINE csubstr get() const { RYML_ASSERT(valid()); return str; }
csubstr str;
size_t reqlen;
};
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
namespace detail {
// is there a better way to do this?
template<int8_t signedval, uint8_t unsignedval>
struct _charconstant_t
: public std::conditional<std::is_signed<char>::value,
std::integral_constant<int8_t, signedval>,
std::integral_constant<int8_t, static_cast<int8_t>(unsignedval)>,
std::integral_constant<uint8_t, unsignedval>>::type
{};
#define _RYML_CHCONST(signedval, unsignedval) ::c4::yml::detail::_charconstant_t<INT8_C(signedval), UINT8_C(unsignedval)>::value
@@ -411,10 +550,11 @@ struct _SubstrWriter
{
substr buf;
size_t pos;
_SubstrWriter(substr buf_, size_t pos_=0) : buf(buf_), pos(pos_) {}
_SubstrWriter(substr buf_, size_t pos_=0) : buf(buf_), pos(pos_) { C4_ASSERT(buf.str); }
void append(csubstr s)
{
C4_ASSERT(!s.overlaps(buf));
C4_ASSERT(s.str || !s.len);
if(s.len && pos + s.len <= buf.len)
{
C4_ASSERT(s.str);
@@ -424,12 +564,14 @@ struct _SubstrWriter
}
void append(char c)
{
C4_ASSERT(buf.str);
if(pos < buf.len)
buf.str[pos] = c;
++pos;
}
void append_n(char c, size_t numtimes)
{
C4_ASSERT(buf.str);
if(numtimes && pos + numtimes < buf.len)
memset(buf.str + pos, c, numtimes);
pos += numtimes;
@@ -439,15 +581,77 @@ struct _SubstrWriter
//! get the part written so far
csubstr curr() const { return pos <= buf.len ? buf.first(pos) : buf; }
//! get the part that is still free to write to (the remainder)
substr rem() { return pos < buf.len ? buf.sub(pos) : buf.last(0); }
substr rem() const { return pos < buf.len ? buf.sub(pos) : buf.last(0); }
size_t advance(size_t more) { pos += more; return pos; }
};
} // namespace detail
namespace detail {
// dumpfn is a function abstracting prints to terminal (or to string).
template<class DumpFn, class ...Args>
C4_NO_INLINE void _dump(DumpFn &&dumpfn, csubstr fmt, Args&& ...args)
{
DumpResults results;
// try writing everything:
{
// buffer for converting individual arguments. it is defined
// in a child scope to free it in case the buffer is too small
// for any of the arguments.
char writebuf[RYML_LOGBUF_SIZE];
results = format_dump_resume(std::forward<DumpFn>(dumpfn), writebuf, fmt, std::forward<Args>(args)...);
}
// if any of the arguments failed to fit the buffer, allocate a
// larger buffer (up to a limit) and resume writing.
//
// results.bufsize is set to the size of the largest element
// serialized. Eg int(1) will require 1 byte.
if(C4_UNLIKELY(results.bufsize > RYML_LOGBUF_SIZE))
{
const size_t bufsize = results.bufsize <= RYML_LOGBUF_SIZE_MAX ? results.bufsize : RYML_LOGBUF_SIZE_MAX;
#ifdef C4_MSVC
substr largerbuf = {static_cast<char*>(_alloca(bufsize)), bufsize};
#else
substr largerbuf = {static_cast<char*>(alloca(bufsize)), bufsize};
#endif
results = format_dump_resume(std::forward<DumpFn>(dumpfn), results, largerbuf, fmt, std::forward<Args>(args)...);
}
}
template<class ...Args>
C4_NORETURN C4_NO_INLINE void _report_err(Callbacks const& C4_RESTRICT callbacks, csubstr fmt, Args const& C4_RESTRICT ...args)
{
char errmsg[RYML_ERRMSG_SIZE] = {0};
detail::_SubstrWriter writer(errmsg);
auto dumpfn = [&writer](csubstr s){ writer.append(s); };
_dump(dumpfn, fmt, args...);
writer.append('\n');
const size_t len = writer.pos < RYML_ERRMSG_SIZE ? writer.pos : RYML_ERRMSG_SIZE;
callbacks.m_error(errmsg, len, {}, callbacks.m_user_data);
C4_UNREACHABLE_AFTER_ERR();
}
} // namespace detail
inline csubstr _c4prc(const char &C4_RESTRICT c) // pass by reference!
{
switch(c)
{
case '\n': return csubstr("\\n");
case '\t': return csubstr("\\t");
case '\0': return csubstr("\\0");
case '\r': return csubstr("\\r");
case '\f': return csubstr("\\f");
case '\b': return csubstr("\\b");
case '\v': return csubstr("\\v");
case '\a': return csubstr("\\a");
default: return csubstr(&c, 1);
}
}
/// @endcond
C4_SUPPRESS_WARNING_GCC_CLANG_POP
C4_SUPPRESS_WARNING_GCC_POP
} // namespace yml
} // namespace c4

View File

@@ -17,7 +17,7 @@ namespace c4 {
namespace yml {
void check_invariants(Tree const& t, size_t node=NONE);
void check_invariants(Tree const& t, id_type node=NONE);
void check_free_list(Tree const& t);
void check_arena(Tree const& t);
@@ -26,16 +26,16 @@ void check_arena(Tree const& t);
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
inline void check_invariants(Tree const& t, size_t node)
inline void check_invariants(Tree const& t, id_type node)
{
if(node == NONE)
{
if(t.size() == 0) return;
if(t.empty()) return;
node = t.root_id();
}
auto const& n = *t._p(node);
#ifdef RYML_DBG
NodeData const& n = *t._p(node);
#if defined(RYML_DBG) && 0
if(n.m_first_child != NONE || n.m_last_child != NONE)
{
printf("check(%zu): fc=%zu lc=%zu\n", node, n.m_first_child, n.m_last_child);
@@ -100,10 +100,10 @@ inline void check_invariants(Tree const& t, size_t node)
C4_CHECK(t._p(n.m_next_sibling)->m_next_sibling != node);
}
size_t count = 0;
for(size_t i = n.m_first_child; i != NONE; i = t.next_sibling(i))
id_type count = 0;
for(id_type i = n.m_first_child; i != NONE; i = t.next_sibling(i))
{
#ifdef RYML_DBG
#if defined(RYML_DBG) && 0
printf("check(%zu): descend to child[%zu]=%zu\n", node, count, i);
#endif
auto const& ch = *t._p(i);
@@ -131,7 +131,7 @@ inline void check_invariants(Tree const& t, size_t node)
check_arena(t);
}
for(size_t i = t.first_child(node); i != NONE; i = t.next_sibling(i))
for(id_type i = t.first_child(node); i != NONE; i = t.next_sibling(i))
{
check_invariants(t, i);
}
@@ -159,8 +159,8 @@ inline void check_free_list(Tree const& t)
//C4_CHECK(head.m_prev_sibling == NONE);
//C4_CHECK(tail.m_next_sibling == NONE);
size_t count = 0;
for(size_t i = t.m_free_head, prev = NONE; i != NONE; i = t._p(i)->m_next_sibling)
id_type count = 0;
for(id_type i = t.m_free_head, prev = NONE; i != NONE; i = t._p(i)->m_next_sibling)
{
auto const& elm = *t._p(i);
if(&elm != &head)

View File

@@ -4,135 +4,144 @@
#ifndef _C4_YML_COMMON_HPP_
#include "../common.hpp"
#endif
#ifdef RYML_DBG
#include <cstdio>
#endif
//-----------------------------------------------------------------------------
// some debugging scaffolds
#if defined(_MSC_VER)
# pragma warning(push)
# pragma warning(disable: 4068/*unknown pragma*/)
#endif
// NOLINTBEGIN(*)
C4_SUPPRESS_WARNING_GCC_CLANG_PUSH
C4_SUPPRESS_WARNING_MSVC_PUSH
C4_SUPPRESS_WARNING_MSVC(4068/*unknown pragma*/)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wunknown-pragmas"
//#pragma GCC diagnostic ignored "-Wpragma-system-header-outside-header"
#pragma GCC system_header
C4_SUPPRESS_WARNING_GCC("-Wunknown-pragmas")
C4_SUPPRESS_WARNING_CLANG("-Wgnu-zero-variadic-macro-arguments")
// NOLINTEND(*)
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Werror"
#pragma clang diagnostic ignored "-Wgnu-zero-variadic-macro-arguments"
// some debugging scaffolds
#ifdef RYML_DBG
#include <c4/dump.hpp>
namespace c4 {
inline void _dbg_dumper(csubstr s) { fwrite(s.str, 1, s.len, stdout); };
template<class ...Args>
void _dbg_printf(c4::csubstr fmt, Args&& ...args)
{
static char writebuf[256];
auto results = c4::format_dump_resume<&_dbg_dumper>(writebuf, fmt, std::forward<Args>(args)...);
// resume writing if the results failed to fit the buffer
if(C4_UNLIKELY(results.bufsize > sizeof(writebuf))) // bufsize will be that of the largest element serialized. Eg int(1), will require 1 byte.
{
results = format_dump_resume<&_dbg_dumper>(results, writebuf, fmt, std::forward<Args>(args)...);
if(C4_UNLIKELY(results.bufsize > sizeof(writebuf)))
{
results = format_dump_resume<&_dbg_dumper>(results, writebuf, fmt, std::forward<Args>(args)...);
}
}
}
} // namespace c4
# define _c4dbgt(fmt, ...) this->_dbg ("{}:{}: " fmt , __FILE__, __LINE__, ## __VA_ARGS__)
# define _c4dbgpf(fmt, ...) _dbg_printf("{}:{}: " fmt "\n", __FILE__, __LINE__, ## __VA_ARGS__)
# define _c4dbgp(msg) _dbg_printf("{}:{}: " msg "\n", __FILE__, __LINE__ )
# define _c4dbgq(msg) _dbg_printf(msg "\n")
#ifndef RYML_DBG
# define _c4err(fmt, ...) \
do { if(c4::is_debugger_attached()) { C4_DEBUG_BREAK(); } \
this->_err("ERROR:\n" "{}:{}: " fmt, __FILE__, __LINE__, ## __VA_ARGS__); } while(0)
#else
this->_err("ERROR: " fmt, ## __VA_ARGS__)
# define _c4dbgt(fmt, ...)
# define _c4dbgpf(fmt, ...)
# define _c4dbgpf_(fmt, ...)
# define _c4dbgp(msg)
# define _c4dbgp_(msg)
# define _c4dbgq(msg)
# define _c4presc(...)
# define _c4prscalar(msg, scalar, keep_newlines)
#else
# define _c4err(fmt, ...) \
do { if(c4::is_debugger_attached()) { C4_DEBUG_BREAK(); } \
this->_err("ERROR: " fmt, ## __VA_ARGS__); } while(0)
#endif
do { RYML_DEBUG_BREAK(); this->_err("ERROR:\n" "{}:{}: " fmt, __FILE__, __LINE__, ## __VA_ARGS__); } while(0)
# define _c4dbgt(fmt, ...) do { if(_dbg_enabled()) { \
this->_dbg ("{}:{}: " fmt , __FILE__, __LINE__, ## __VA_ARGS__); } } while(0)
# define _c4dbgpf(fmt, ...) _dbg_printf("{}:{}: " fmt "\n", __FILE__, __LINE__, ## __VA_ARGS__)
# define _c4dbgpf_(fmt, ...) _dbg_printf("{}:{}: " fmt , __FILE__, __LINE__, ## __VA_ARGS__)
# define _c4dbgp(msg) _dbg_printf("{}:{}: " msg "\n", __FILE__, __LINE__ )
# define _c4dbgp_(msg) _dbg_printf("{}:{}: " msg , __FILE__, __LINE__ )
# define _c4dbgq(msg) _dbg_printf(msg "\n")
# define _c4presc(...) do { if(_dbg_enabled()) __c4presc(__VA_ARGS__); } while(0)
# define _c4prscalar(msg, scalar, keep_newlines) \
do { \
_c4dbgpf_("{}: [{}]~~~", msg, scalar.len); \
if(_dbg_enabled()) { \
__c4presc((scalar).str, (scalar).len, (keep_newlines)); \
} \
_c4dbgq("~~~"); \
} while(0)
#endif // RYML_DBG
#define _c4prsp(sp) sp
#define _c4presc(s) __c4presc(s.str, s.len)
inline c4::csubstr _c4prc(const char &C4_RESTRICT c)
//-----------------------------------------------------------------------------
#ifdef RYML_DBG
#include <c4/dump.hpp>
namespace c4 {
inline bool& _dbg_enabled() { static bool enabled = true; return enabled; }
inline void _dbg_set_enabled(bool yes) { _dbg_enabled() = yes; }
inline void _dbg_dumper(csubstr s)
{
switch(c)
if(s.str)
fwrite(s.str, 1, s.len, stdout);
}
inline substr _dbg_buf() noexcept
{
static char writebuf[2048];
return substr{writebuf, sizeof(writebuf)}; // g++-5 has trouble with return writebuf;
}
template<class ...Args>
C4_NO_INLINE void _dbg_printf(c4::csubstr fmt, Args const& ...args)
{
if(_dbg_enabled())
{
case '\n': return c4::csubstr("\\n");
case '\t': return c4::csubstr("\\t");
case '\0': return c4::csubstr("\\0");
case '\r': return c4::csubstr("\\r");
case '\f': return c4::csubstr("\\f");
case '\b': return c4::csubstr("\\b");
case '\v': return c4::csubstr("\\v");
case '\a': return c4::csubstr("\\a");
default: return c4::csubstr(&c, 1);
substr buf = _dbg_buf();
const size_t needed_size = c4::format_dump(&_dbg_dumper, buf, fmt, args...);
C4_CHECK(needed_size <= buf.len);
}
}
inline void __c4presc(const char *s, size_t len)
inline C4_NO_INLINE void __c4presc(const char *s, size_t len, bool keep_newlines=false)
{
RYML_ASSERT(s || !len);
size_t prev = 0;
for(size_t i = 0; i < len; ++i)
{
switch(s[i])
{
case '\n' : if(i > prev) { fwrite(s+prev, 1, i-prev, stdout); } putchar('\\'); putchar('n'); putchar('\n'); prev = i+1; break;
case '\t' : if(i > prev) { fwrite(s+prev, 1, i-prev, stdout); } putchar('\\'); putchar('t'); prev = i+1; break;
case '\0' : if(i > prev) { fwrite(s+prev, 1, i-prev, stdout); } putchar('\\'); putchar('0'); prev = i+1; break;
case '\r' : if(i > prev) { fwrite(s+prev, 1, i-prev, stdout); } putchar('\\'); putchar('r'); prev = i+1; break;
case '\f' : if(i > prev) { fwrite(s+prev, 1, i-prev, stdout); } putchar('\\'); putchar('f'); prev = i+1; break;
case '\b' : if(i > prev) { fwrite(s+prev, 1, i-prev, stdout); } putchar('\\'); putchar('b'); prev = i+1; break;
case '\v' : if(i > prev) { fwrite(s+prev, 1, i-prev, stdout); } putchar('\\'); putchar('v'); prev = i+1; break;
case '\a' : if(i > prev) { fwrite(s+prev, 1, i-prev, stdout); } putchar('\\'); putchar('a'); prev = i+1; break;
case '\x1b': if(i > prev) { fwrite(s+prev, 1, i-prev, stdout); } putchar('\\'); putchar('e'); prev = i+1; break;
case '\n' : _dbg_printf("{}{}{}", csubstr(s+prev, i-prev), csubstr("\\n"), csubstr(keep_newlines ? "\n":"")); prev = i+1; break;
case '\t' : _dbg_printf("{}{}", csubstr(s+prev, i-prev), csubstr("\\t")); prev = i+1; break;
case '\0' : _dbg_printf("{}{}", csubstr(s+prev, i-prev), csubstr("\\0")); prev = i+1; break;
case '\r' : _dbg_printf("{}{}", csubstr(s+prev, i-prev), csubstr("\\r")); prev = i+1; break;
case '\f' : _dbg_printf("{}{}", csubstr(s+prev, i-prev), csubstr("\\f")); prev = i+1; break;
case '\b' : _dbg_printf("{}{}", csubstr(s+prev, i-prev), csubstr("\\b")); prev = i+1; break;
case '\v' : _dbg_printf("{}{}", csubstr(s+prev, i-prev), csubstr("\\v")); prev = i+1; break;
case '\a' : _dbg_printf("{}{}", csubstr(s+prev, i-prev), csubstr("\\a")); prev = i+1; break;
case '\x1b': _dbg_printf("{}{}", csubstr(s+prev, i-prev), csubstr("\\x1b")); prev = i+1; break;
case -0x3e/*0xc2u*/:
if(i+1 < len)
{
if(s[i+1] == -0x60/*0xa0u*/)
{
if(i > prev) { fwrite(s+prev, 1, i-prev, stdout); } putchar('\\'); putchar('_'); prev = i+2; ++i;
_dbg_printf("{}{}", csubstr(s+prev, i-prev), csubstr("\\_")); prev = i+1;
}
else if(s[i+1] == -0x7b/*0x85u*/)
{
if(i > prev) { fwrite(s+prev, 1, i-prev, stdout); } putchar('\\'); putchar('N'); prev = i+2; ++i;
_dbg_printf("{}{}", csubstr(s+prev, i-prev), csubstr("\\N")); prev = i+1;
}
break;
}
break;
case -0x1e/*0xe2u*/:
if(i+2 < len && s[i+1] == -0x80/*0x80u*/)
{
if(s[i+2] == -0x58/*0xa8u*/)
{
if(i > prev) { fwrite(s+prev, 1, i-prev, stdout); } putchar('\\'); putchar('L'); prev = i+3; i += 2;
_dbg_printf("{}{}", csubstr(s+prev, i-prev), csubstr("\\L")); prev = i+1;
}
else if(s[i+2] == -0x57/*0xa9u*/)
{
if(i > prev) { fwrite(s+prev, 1, i-prev, stdout); } putchar('\\'); putchar('P'); prev = i+3; i += 2;
_dbg_printf("{}{}", csubstr(s+prev, i-prev), csubstr("\\P")); prev = i+1;
}
break;
}
break;
}
}
if(len > prev)
fwrite(s + prev, 1, len - prev, stdout);
_dbg_printf("{}", csubstr(s+prev, len-prev));
}
inline void __c4presc(csubstr s, bool keep_newlines=false)
{
__c4presc(s.str, s.len, keep_newlines);
}
} // namespace c4
#pragma clang diagnostic pop
#pragma GCC diagnostic pop
#if defined(_MSC_VER)
# pragma warning(pop)
#endif
#endif // RYML_DBG
C4_SUPPRESS_WARNING_GCC_CLANG_POP
C4_SUPPRESS_WARNING_MSVC_POP
#endif /* _C4_YML_DETAIL_PARSER_DBG_HPP_ */

View File

@@ -4,20 +4,76 @@
#include "c4/yml/tree.hpp"
#include "c4/yml/node.hpp"
#ifdef RYML_DBG
#define _c4dbg_tree(...) print_tree(__VA_ARGS__)
#define _c4dbg_node(...) print_tree(__VA_ARGS__)
#else
#define _c4dbg_tree(...)
#define _c4dbg_node(...)
#endif
namespace c4 {
namespace yml {
C4_SUPPRESS_WARNING_GCC_CLANG_WITH_PUSH("-Wold-style-cast")
C4_SUPPRESS_WARNING_GCC("-Wuseless-cast")
inline size_t print_node(Tree const& p, size_t node, int level, size_t count, bool print_children)
inline const char* _container_style_code(Tree const& p, id_type node)
{
printf("[%zd]%*s[%zd] %p", count, (2*level), "", node, (void const*)p.get(node));
if(p.is_container(node))
{
if(p._p(node)->m_type & (FLOW_SL|FLOW_ML))
{
return "[FLOW]";
}
if(p._p(node)->m_type & (BLOCK))
{
return "[BLCK]";
}
}
return "";
}
inline char _scalar_code(NodeType masked)
{
if(masked & (KEY_LITERAL|VAL_LITERAL))
return '|';
if(masked & (KEY_FOLDED|VAL_FOLDED))
return '>';
if(masked & (KEY_SQUO|VAL_SQUO))
return '\'';
if(masked & (KEY_DQUO|VAL_DQUO))
return '"';
if(masked & (KEY_PLAIN|VAL_PLAIN))
return '~';
return '@';
}
inline char _scalar_code_key(NodeType t)
{
return _scalar_code(t & KEY_STYLE);
}
inline char _scalar_code_val(NodeType t)
{
return _scalar_code(t & VAL_STYLE);
}
inline char _scalar_code_key(Tree const& p, id_type node)
{
return _scalar_code_key(p._p(node)->m_type);
}
inline char _scalar_code_val(Tree const& p, id_type node)
{
return _scalar_code_key(p._p(node)->m_type);
}
inline id_type print_node(Tree const& p, id_type node, int level, id_type count, bool print_children)
{
printf("[%zu]%*s[%zu] %p", (size_t)count, (2*level), "", (size_t)node, (void const*)p.get(node));
if(p.is_root(node))
{
printf(" [ROOT]");
}
printf(" %s:", p.type_str(node));
char typebuf[128];
csubstr typestr = p.type(node).type_str(typebuf);
RYML_CHECK(typestr.str);
printf(" %.*s", (int)typestr.len, typestr.str);
if(p.has_key(node))
{
if(p.has_key_anchor(node))
@@ -28,65 +84,47 @@ inline size_t print_node(Tree const& p, size_t node, int level, size_t count, bo
if(p.has_key_tag(node))
{
csubstr kt = p.key_tag(node);
csubstr k = p.key(node);
printf(" %.*s '%.*s'", (int)kt.len, kt.str, (int)k.len, k.str);
}
else
{
csubstr k = p.key(node);
printf(" '%.*s'", (int)k.len, k.str);
}
}
else
{
RYML_ASSERT( ! p.has_key_tag(node));
}
if(p.has_val(node))
{
if(p.has_val_tag(node))
{
csubstr vt = p.val_tag(node);
csubstr v = p.val(node);
printf(" %.*s '%.*s'", (int)vt.len, vt.str, (int)v.len, v.str);
}
else
{
csubstr v = p.val(node);
printf(" '%.*s'", (int)v.len, v.str);
}
}
else
{
if(p.has_val_tag(node))
{
csubstr vt = p.val_tag(node);
printf(" %.*s", (int)vt.len, vt.str);
printf(" <%.*s>", (int)kt.len, kt.str);
}
const char code = _scalar_code_key(p, node);
csubstr k = p.key(node);
printf(" %c%.*s%c :", code, (int)k.len, k.str, code);
}
if(p.has_val_anchor(node))
{
auto &a = p.val_anchor(node);
printf(" valanchor='&%.*s'", (int)a.len, a.str);
csubstr a = p.val_anchor(node);
printf(" &%.*s'", (int)a.len, a.str);
}
printf(" (%zd sibs)", p.num_siblings(node));
if(p.has_val_tag(node))
{
csubstr vt = p.val_tag(node);
printf(" <%.*s>", (int)vt.len, vt.str);
}
if(p.has_val(node))
{
const char code = _scalar_code_val(p, node);
csubstr v = p.val(node);
printf(" %c%.*s%c", code, (int)v.len, v.str, code);
}
printf(" (%zu sibs)", (size_t)p.num_siblings(node));
++count;
if(p.is_container(node))
if(!p.is_container(node))
{
printf(" %zd children:\n", p.num_children(node));
printf("\n");
}
else
{
printf(" (%zu children)\n", (size_t)p.num_children(node));
if(print_children)
{
for(size_t i = p.first_child(node); i != NONE; i = p.next_sibling(i))
for(id_type i = p.first_child(node); i != NONE; i = p.next_sibling(i))
{
count = print_node(p, i, level+1, count, print_children);
}
}
}
else
{
printf("\n");
}
return count;
}
@@ -106,21 +144,37 @@ inline void print_node(ConstNodeRef const& p, int level=0)
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
inline size_t print_tree(Tree const& p, size_t node=NONE)
inline id_type print_tree(const char *message, Tree const& p, id_type node=NONE)
{
printf("--------------------------------------\n");
size_t ret = 0;
if(message != nullptr)
printf("%s:\n", message);
id_type ret = 0;
if(!p.empty())
{
if(node == NONE)
node = p.root_id();
ret = print_node(p, node, 0, 0, true);
}
printf("#nodes=%zd vs #printed=%zd\n", p.size(), ret);
printf("#nodes=%zu vs #printed=%zu\n", (size_t)p.size(), (size_t)ret);
printf("--------------------------------------\n");
return ret;
}
inline id_type print_tree(Tree const& p, id_type node=NONE)
{
return print_tree(nullptr, p, node);
}
inline void print_tree(ConstNodeRef const& p, int level)
{
print_node(p, level);
for(ConstNodeRef ch : p.children())
{
print_tree(ch, level+1);
}
}
C4_SUPPRESS_WARNING_GCC_CLANG_POP
} /* namespace yml */

View File

@@ -18,22 +18,27 @@ C4_SUPPRESS_WARNING_GCC_CLANG_WITH_PUSH("-Wold-style-cast")
namespace detail {
/** A lightweight contiguous stack with SSO. This avoids a dependency on std. */
template<class T, size_t N=16>
/** A lightweight contiguous stack with Small Storage
* Optimization. This is required because std::vector can throw
* exceptions, and we don't want to enforce any particular error
* mechanism. */
template<class T, id_type N=16>
class stack
{
static_assert(std::is_trivially_copyable<T>::value, "T must be trivially copyable");
static_assert(std::is_trivially_destructible<T>::value, "T must be trivially destructible");
enum : size_t { sso_size = N };
public:
enum : id_type { sso_size = N };
public:
T m_buf[N];
T * m_stack;
size_t m_size;
size_t m_capacity;
Callbacks m_callbacks;
T m_buf[size_t(N)];
T *C4_RESTRICT m_stack;
id_type m_size;
id_type m_capacity;
Callbacks m_callbacks;
public:
@@ -64,9 +69,12 @@ public:
stack& operator= (stack const& that) RYML_NOEXCEPT
{
_cb(that.m_callbacks);
resize(that.m_size);
_cp(&that);
if(&that != this)
{
_cb(that.m_callbacks);
resize(that.m_size);
_cp(&that);
}
return *this;
}
@@ -79,29 +87,29 @@ public:
public:
size_t size() const { return m_size; }
size_t empty() const { return m_size == 0; }
size_t capacity() const { return m_capacity; }
id_type size() const { return m_size; }
id_type empty() const { return m_size == 0; }
id_type capacity() const { return m_capacity; }
void clear()
{
m_size = 0;
}
void resize(size_t sz)
void resize(id_type sz)
{
reserve(sz);
m_size = sz;
}
void reserve(size_t sz);
void reserve(id_type sz);
void push(T const& C4_RESTRICT n)
{
RYML_ASSERT((const char*)&n + sizeof(T) < (const char*)m_stack || &n > m_stack + m_capacity);
_RYML_CB_ASSERT(m_callbacks, (const char*)&n + sizeof(T) < (const char*)m_stack || &n > m_stack + m_capacity);
if(m_size == m_capacity)
{
size_t cap = m_capacity == 0 ? N : 2 * m_capacity;
id_type cap = m_capacity == 0 ? N : 2 * m_capacity;
reserve(cap);
}
m_stack[m_size] = n;
@@ -110,10 +118,10 @@ public:
void push_top()
{
RYML_ASSERT(m_size > 0);
_RYML_CB_ASSERT(m_callbacks, m_size > 0);
if(m_size == m_capacity)
{
size_t cap = m_capacity == 0 ? N : 2 * m_capacity;
id_type cap = m_capacity == 0 ? N : 2 * m_capacity;
reserve(cap);
}
m_stack[m_size] = m_stack[m_size - 1];
@@ -122,25 +130,25 @@ public:
T const& C4_RESTRICT pop()
{
RYML_ASSERT(m_size > 0);
_RYML_CB_ASSERT(m_callbacks, m_size > 0);
--m_size;
return m_stack[m_size];
}
C4_ALWAYS_INLINE T const& C4_RESTRICT top() const { RYML_ASSERT(m_size > 0); return m_stack[m_size - 1]; }
C4_ALWAYS_INLINE T & C4_RESTRICT top() { RYML_ASSERT(m_size > 0); return m_stack[m_size - 1]; }
C4_ALWAYS_INLINE T const& C4_RESTRICT top() const { _RYML_CB_ASSERT(m_callbacks, m_size > 0); return m_stack[m_size - 1]; }
C4_ALWAYS_INLINE T & C4_RESTRICT top() { _RYML_CB_ASSERT(m_callbacks, m_size > 0); return m_stack[m_size - 1]; }
C4_ALWAYS_INLINE T const& C4_RESTRICT bottom() const { RYML_ASSERT(m_size > 0); return m_stack[0]; }
C4_ALWAYS_INLINE T & C4_RESTRICT bottom() { RYML_ASSERT(m_size > 0); return m_stack[0]; }
C4_ALWAYS_INLINE T const& C4_RESTRICT bottom() const { _RYML_CB_ASSERT(m_callbacks, m_size > 0); return m_stack[0]; }
C4_ALWAYS_INLINE T & C4_RESTRICT bottom() { _RYML_CB_ASSERT(m_callbacks, m_size > 0); return m_stack[0]; }
C4_ALWAYS_INLINE T const& C4_RESTRICT top(size_t i) const { RYML_ASSERT(i < m_size); return m_stack[m_size - 1 - i]; }
C4_ALWAYS_INLINE T & C4_RESTRICT top(size_t i) { RYML_ASSERT(i < m_size); return m_stack[m_size - 1 - i]; }
C4_ALWAYS_INLINE T const& C4_RESTRICT top(id_type i) const { _RYML_CB_ASSERT(m_callbacks, i < m_size); return m_stack[m_size - 1 - i]; }
C4_ALWAYS_INLINE T & C4_RESTRICT top(id_type i) { _RYML_CB_ASSERT(m_callbacks, i < m_size); return m_stack[m_size - 1 - i]; }
C4_ALWAYS_INLINE T const& C4_RESTRICT bottom(size_t i) const { RYML_ASSERT(i < m_size); return m_stack[i]; }
C4_ALWAYS_INLINE T & C4_RESTRICT bottom(size_t i) { RYML_ASSERT(i < m_size); return m_stack[i]; }
C4_ALWAYS_INLINE T const& C4_RESTRICT bottom(id_type i) const { _RYML_CB_ASSERT(m_callbacks, i < m_size); return m_stack[i]; }
C4_ALWAYS_INLINE T & C4_RESTRICT bottom(id_type i) { _RYML_CB_ASSERT(m_callbacks, i < m_size); return m_stack[i]; }
C4_ALWAYS_INLINE T const& C4_RESTRICT operator[](size_t i) const { RYML_ASSERT(i < m_size); return m_stack[i]; }
C4_ALWAYS_INLINE T & C4_RESTRICT operator[](size_t i) { RYML_ASSERT(i < m_size); return m_stack[i]; }
C4_ALWAYS_INLINE T const& C4_RESTRICT operator[](id_type i) const { _RYML_CB_ASSERT(m_callbacks, i < m_size); return m_stack[i]; }
C4_ALWAYS_INLINE T & C4_RESTRICT operator[](id_type i) { _RYML_CB_ASSERT(m_callbacks, i < m_size); return m_stack[i]; }
public:
@@ -154,10 +162,12 @@ public:
const_iterator end () const { return (const_iterator)m_stack + m_size; }
public:
void _free();
void _cp(stack const* C4_RESTRICT that);
void _mv(stack * that);
void _cb(Callbacks const& cb);
};
@@ -165,8 +175,8 @@ public:
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
template<class T, size_t N>
void stack<T, N>::reserve(size_t sz)
template<class T, id_type N>
void stack<T, N>::reserve(id_type sz)
{
if(sz <= m_size)
return;
@@ -176,11 +186,12 @@ void stack<T, N>::reserve(size_t sz)
m_capacity = N;
return;
}
T *buf = (T*) m_callbacks.m_allocate(sz * sizeof(T), m_stack, m_callbacks.m_user_data);
memcpy(buf, m_stack, m_size * sizeof(T));
T *buf = (T*) m_callbacks.m_allocate((size_t)sz * sizeof(T), m_stack, m_callbacks.m_user_data);
_RYML_CB_ASSERT(m_callbacks, ((uintptr_t)buf % alignof(T)) == 0u);
memcpy(buf, m_stack, (size_t)m_size * sizeof(T));
if(m_stack != m_buf)
{
m_callbacks.m_free(m_stack, m_capacity * sizeof(T), m_callbacks.m_user_data);
m_callbacks.m_free(m_stack, (size_t)m_capacity * sizeof(T), m_callbacks.m_user_data);
}
m_stack = buf;
m_capacity = sz;
@@ -189,38 +200,38 @@ void stack<T, N>::reserve(size_t sz)
//-----------------------------------------------------------------------------
template<class T, size_t N>
template<class T, id_type N>
void stack<T, N>::_free()
{
RYML_ASSERT(m_stack != nullptr); // this structure cannot be memset() to zero
_RYML_CB_ASSERT(m_callbacks, m_stack != nullptr); // this structure cannot be memset() to zero
if(m_stack != m_buf)
{
m_callbacks.m_free(m_stack, m_capacity * sizeof(T), m_callbacks.m_user_data);
m_callbacks.m_free(m_stack, (size_t)m_capacity * sizeof(T), m_callbacks.m_user_data);
m_stack = m_buf;
m_size = N;
m_capacity = N;
}
else
{
RYML_ASSERT(m_capacity == N);
_RYML_CB_ASSERT(m_callbacks, m_capacity == N);
}
}
//-----------------------------------------------------------------------------
template<class T, size_t N>
template<class T, id_type N>
void stack<T, N>::_cp(stack const* C4_RESTRICT that)
{
if(that->m_stack != that->m_buf)
{
RYML_ASSERT(that->m_capacity > N);
RYML_ASSERT(that->m_size <= that->m_capacity);
_RYML_CB_ASSERT(m_callbacks, that->m_capacity > N);
_RYML_CB_ASSERT(m_callbacks, that->m_size <= that->m_capacity);
}
else
{
RYML_ASSERT(that->m_capacity <= N);
RYML_ASSERT(that->m_size <= that->m_capacity);
_RYML_CB_ASSERT(m_callbacks, that->m_capacity <= N);
_RYML_CB_ASSERT(m_callbacks, that->m_size <= that->m_capacity);
}
memcpy(m_stack, that->m_stack, that->m_size * sizeof(T));
m_size = that->m_size;
@@ -231,19 +242,19 @@ void stack<T, N>::_cp(stack const* C4_RESTRICT that)
//-----------------------------------------------------------------------------
template<class T, size_t N>
template<class T, id_type N>
void stack<T, N>::_mv(stack * that)
{
if(that->m_stack != that->m_buf)
{
RYML_ASSERT(that->m_capacity > N);
RYML_ASSERT(that->m_size <= that->m_capacity);
_RYML_CB_ASSERT(m_callbacks, that->m_capacity > N);
_RYML_CB_ASSERT(m_callbacks, that->m_size <= that->m_capacity);
m_stack = that->m_stack;
}
else
{
RYML_ASSERT(that->m_capacity <= N);
RYML_ASSERT(that->m_size <= that->m_capacity);
_RYML_CB_ASSERT(m_callbacks, that->m_capacity <= N);
_RYML_CB_ASSERT(m_callbacks, that->m_size <= that->m_capacity);
memcpy(m_buf, that->m_buf, that->m_size * sizeof(T));
m_stack = m_buf;
}
@@ -251,7 +262,7 @@ void stack<T, N>::_mv(stack * that)
m_capacity = that->m_capacity;
m_callbacks = that->m_callbacks;
// make sure no deallocation happens on destruction
RYML_ASSERT(that->m_stack != m_buf);
_RYML_CB_ASSERT(m_callbacks, that->m_stack != m_buf);
that->m_stack = that->m_buf;
that->m_capacity = N;
that->m_size = 0;
@@ -260,7 +271,7 @@ void stack<T, N>::_mv(stack * that)
//-----------------------------------------------------------------------------
template<class T, size_t N>
template<class T, id_type N>
void stack<T, N>::_cb(Callbacks const& cb)
{
if(cb != m_callbacks)

File diff suppressed because it is too large Load Diff

View File

@@ -15,13 +15,8 @@
#include "./node.hpp"
#endif
#ifdef emit
#error "emit is defined, likely from a Qt include. This will cause a compilation error. See https://github.com/biojppm/rapidyaml/issues/120"
#endif
#define RYML_DEPRECATE_EMIT \
RYML_DEPRECATED("use emit_yaml() instead. See https://github.com/biojppm/rapidyaml/issues/120")
#define RYML_DEPRECATE_EMITRS \
RYML_DEPRECATED("use emitrs_yaml() instead. See https://github.com/biojppm/rapidyaml/issues/120")
C4_SUPPRESS_WARNING_GCC_CLANG_WITH_PUSH("-Wold-style-cast")
//-----------------------------------------------------------------------------
@@ -43,6 +38,15 @@ using EmitterOStream = Emitter<WriterOStream<OStream>>;
using EmitterFile = Emitter<WriterFile>;
using EmitterBuf = Emitter<WriterBuf>;
namespace detail {
inline bool is_set_(ConstNodeRef n) { return n.tree() && (n.id() != NONE); }
}
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
/** Specifies the type of content to emit */
typedef enum {
EMIT_YAML = 0, ///< emit YAML
@@ -50,6 +54,59 @@ typedef enum {
} EmitType_e;
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
/** A lightweight object containing options to be used when emitting. */
struct EmitOptions
{
typedef enum : uint32_t {
DEFAULT_FLAGS = 0u,
JSON_ERR_ON_TAG = 1u << 0u,
JSON_ERR_ON_ANCHOR = 1u << 1u,
_JSON_ERR_MASK = JSON_ERR_ON_TAG|JSON_ERR_ON_ANCHOR,
} EmitOptionFlags_e;
public:
/** @name option flags
*
* @{ */
C4_ALWAYS_INLINE EmitOptionFlags_e json_error_flags() const noexcept { return m_option_flags; }
EmitOptions& json_error_flags(EmitOptionFlags_e d) noexcept { m_option_flags = (EmitOptionFlags_e)(d & _JSON_ERR_MASK); return *this; }
/** @} */
public:
/** @name max depth for the emitted tree
*
* This makes the emitter fail when emitting trees exceeding the
* max_depth.
*
* @{ */
C4_ALWAYS_INLINE id_type max_depth() const noexcept { return m_max_depth; }
EmitOptions& max_depth(id_type d) noexcept { m_max_depth = d; return *this; }
static constexpr const id_type max_depth_default = 64;
/** @} */
public:
bool operator== (const EmitOptions& that) const noexcept
{
return m_max_depth == that.m_max_depth &&
m_option_flags == that.m_option_flags;
}
private:
/** @cond dev */
id_type m_max_depth{max_depth_default};
EmitOptionFlags_e m_option_flags{DEFAULT_FLAGS};
/** @endcond */
};
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
@@ -61,12 +118,21 @@ class Emitter : public Writer
{
public:
/** Construct the emitter and its internal Writer state. Every
* parameter is forwarded to the constructor of the writer. */
/** Construct the emitter and its internal Writer state, using default emit options.
* @param args arguments to be forwarded to the constructor of the writer.
* */
template<class ...Args>
Emitter(Args &&...args) : Writer(std::forward<Args>(args)...), m_tree() {}
/** emit!
Emitter(Args &&...args) : Writer(std::forward<Args>(args)...), m_tree(), m_opts(), m_flow(false) {}
/** Construct the emitter and its internal Writer state.
*
* @param opts EmitOptions
* @param args arguments to be forwarded to the constructor of the writer.
* */
template<class ...Args>
Emitter(EmitOptions const& opts, Args &&...args) : Writer(std::forward<Args>(args)...), m_tree(), m_opts(opts), m_flow(false) {}
/** emit!
*
* When writing to a buffer, returns a substr of the emitted YAML.
* If the given buffer has insufficient space, the returned substr
@@ -83,36 +149,63 @@ public:
* @param error_on_excess when true, an error is raised when the
* output buffer is too small for the emitted YAML/JSON
* */
substr emit_as(EmitType_e type, Tree const& t, size_t id, bool error_on_excess);
substr emit_as(EmitType_e type, Tree const& t, id_type id, bool error_on_excess);
/** emit starting at the root node */
substr emit_as(EmitType_e type, Tree const& t, bool error_on_excess=true);
/** emit the given node */
substr emit_as(EmitType_e type, ConstNodeRef const& n, bool error_on_excess=true);
substr emit_as(EmitType_e type, Tree const& t, bool error_on_excess=true)
{
if(t.empty())
return {};
return this->emit_as(type, t, t.root_id(), error_on_excess);
}
/** emit starting at the given node */
substr emit_as(EmitType_e type, ConstNodeRef const& n, bool error_on_excess=true)
{
if(!detail::is_set_(n))
return {};
_RYML_CB_CHECK(n.tree()->callbacks(), n.readable());
return this->emit_as(type, *n.tree(), n.id(), error_on_excess);
}
public:
/** get the emit options for this object */
EmitOptions const& options() const noexcept { return m_opts; }
/** set the max depth for emitted trees (to prevent a stack overflow) */
void max_depth(id_type max_depth) noexcept { m_opts.max_depth(max_depth); }
/** get the max depth for emitted trees (to prevent a stack overflow) */
id_type max_depth() const noexcept { return m_opts.max_depth(); }
private:
Tree const* C4_RESTRICT m_tree;
void _emit_yaml(size_t id);
void _do_visit_flow_sl(size_t id, size_t ilevel=0);
void _do_visit_flow_ml(size_t id, size_t ilevel=0, size_t do_indent=1);
void _do_visit_block(size_t id, size_t ilevel=0, size_t do_indent=1);
void _do_visit_block_container(size_t id, size_t next_level, size_t do_indent);
void _do_visit_json(size_t id);
EmitOptions m_opts;
bool m_flow;
private:
void _write(NodeScalar const& C4_RESTRICT sc, NodeType flags, size_t level);
void _emit_yaml(id_type id);
void _do_visit_flow_sl(id_type id, id_type depth, id_type ilevel=0);
void _do_visit_flow_ml(id_type id, id_type depth, id_type ilevel=0, id_type do_indent=1);
void _do_visit_block(id_type id, id_type depth, id_type ilevel=0, id_type do_indent=1);
void _do_visit_block_container(id_type id, id_type depth, id_type next_level, bool do_indent);
void _do_visit_json(id_type id, id_type depth);
private:
void _write(NodeScalar const& C4_RESTRICT sc, NodeType flags, id_type level);
void _write_json(NodeScalar const& C4_RESTRICT sc, NodeType flags);
void _write_doc(size_t id);
void _write_scalar(csubstr s, bool was_quoted);
void _write_scalar_json(csubstr s, bool as_key, bool was_quoted);
void _write_scalar_literal(csubstr s, size_t level, bool as_key, bool explicit_indentation=false);
void _write_scalar_folded(csubstr s, size_t level, bool as_key);
void _write_scalar_squo(csubstr s, size_t level);
void _write_scalar_dquo(csubstr s, size_t level);
void _write_scalar_plain(csubstr s, size_t level);
void _write_doc(id_type id);
void _write_scalar_json_dquo(csubstr s);
void _write_scalar_literal(csubstr s, id_type level, bool as_key);
void _write_scalar_folded(csubstr s, id_type level, bool as_key);
void _write_scalar_squo(csubstr s, id_type level);
void _write_scalar_dquo(csubstr s, id_type level);
void _write_scalar_plain(csubstr s, id_type level);
size_t _write_escaped_newlines(csubstr s, size_t i);
size_t _write_indented_block(csubstr s, size_t i, id_type level);
void _write_tag(csubstr tag)
{
@@ -122,18 +215,28 @@ private:
}
enum : type_bits {
_keysc = (KEY|KEYREF|KEYANCH|KEYQUO|_WIP_KEY_STYLE) | ~(VAL|VALREF|VALANCH|VALQUO|_WIP_VAL_STYLE),
_valsc = ~(KEY|KEYREF|KEYANCH|KEYQUO|_WIP_KEY_STYLE) | (VAL|VALREF|VALANCH|VALQUO|_WIP_VAL_STYLE),
_keysc = (KEY|KEYREF|KEYANCH|KEYQUO|KEY_STYLE) | ~(VAL|VALREF|VALANCH|VALQUO|VAL_STYLE) | CONTAINER_STYLE,
_valsc = ~(KEY|KEYREF|KEYANCH|KEYQUO|KEY_STYLE) | (VAL|VALREF|VALANCH|VALQUO|VAL_STYLE) | CONTAINER_STYLE,
_keysc_json = (KEY) | ~(VAL),
_valsc_json = ~(KEY) | (VAL),
};
C4_ALWAYS_INLINE void _writek(size_t id, size_t level) { _write(m_tree->keysc(id), m_tree->_p(id)->m_type.type & ~_valsc, level); }
C4_ALWAYS_INLINE void _writev(size_t id, size_t level) { _write(m_tree->valsc(id), m_tree->_p(id)->m_type.type & ~_keysc, level); }
C4_ALWAYS_INLINE void _writek(id_type id, id_type level) { _write(m_tree->keysc(id), (m_tree->_p(id)->m_type.type & ~_valsc), level); }
C4_ALWAYS_INLINE void _writev(id_type id, id_type level) { _write(m_tree->valsc(id), (m_tree->_p(id)->m_type.type & ~_keysc), level); }
C4_ALWAYS_INLINE void _writek_json(size_t id) { _write_json(m_tree->keysc(id), m_tree->_p(id)->m_type.type & ~(VAL)); }
C4_ALWAYS_INLINE void _writev_json(size_t id) { _write_json(m_tree->valsc(id), m_tree->_p(id)->m_type.type & ~(KEY)); }
C4_ALWAYS_INLINE void _writek_json(id_type id) { _write_json(m_tree->keysc(id), m_tree->_p(id)->m_type.type & ~(VAL)); }
C4_ALWAYS_INLINE void _writev_json(id_type id) { _write_json(m_tree->valsc(id), m_tree->_p(id)->m_type.type & ~(KEY)); }
void _indent(id_type level, bool enabled)
{
if(enabled)
this->Writer::_do_write(' ', 2u * (size_t)level);
}
void _indent(id_type level)
{
if(!m_flow)
this->Writer::_do_write(' ', 2u * (size_t)level);
}
};
@@ -147,33 +250,59 @@ private:
*/
/** emit YAML to the given file. A null file defaults to stdout.
* Return the number of bytes written. */
inline size_t emit_yaml(Tree const& t, size_t id, FILE *f)
// emit from tree and node id -----------------------
/** (1) emit YAML to the given file, starting at the given node. A null
* file defaults to stdout. Return the number of bytes written. */
inline size_t emit_yaml(Tree const& t, id_type id, EmitOptions const& opts, FILE *f)
{
EmitterFile em(opts, f);
return em.emit_as(EMIT_YAML, t, id, /*error_on_excess*/true).len;
}
/** (2) like (1), but use default emit options */
inline size_t emit_yaml(Tree const& t, id_type id, FILE *f)
{
EmitterFile em(f);
return em.emit_as(EMIT_YAML, t, id, /*error_on_excess*/true).len;
}
/** emit JSON to the given file. A null file defaults to stdout.
* Return the number of bytes written. */
inline size_t emit_json(Tree const& t, size_t id, FILE *f)
/** (1) emit JSON to the given file, starting at the given node. A null
* file defaults to stdout. Return the number of bytes written. */
inline size_t emit_json(Tree const& t, id_type id, EmitOptions const& opts, FILE *f)
{
EmitterFile em(opts, f);
return em.emit_as(EMIT_JSON, t, id, /*error_on_excess*/true).len;
}
/** (2) like (1), but use default emit options */
inline size_t emit_json(Tree const& t, id_type id, FILE *f)
{
EmitterFile em(f);
return em.emit_as(EMIT_JSON, t, id, /*error_on_excess*/true).len;
}
/** emit YAML to the given file. A null file defaults to stdout.
* Return the number of bytes written.
* @overload */
// emit from root -------------------------
/** (1) emit YAML to the given file, starting at the root node. A null file defaults to stdout.
* Return the number of bytes written. */
inline size_t emit_yaml(Tree const& t, EmitOptions const& opts, FILE *f=nullptr)
{
EmitterFile em(opts, f);
return em.emit_as(EMIT_YAML, t, /*error_on_excess*/true).len;
}
/** (2) like (1), but use default emit options */
inline size_t emit_yaml(Tree const& t, FILE *f=nullptr)
{
EmitterFile em(f);
return em.emit_as(EMIT_YAML, t, /*error_on_excess*/true).len;
}
/** emit JSON to the given file. A null file defaults to stdout.
* Return the number of bytes written.
* @overload */
/** (1) emit JSON to the given file. A null file defaults to stdout.
* Return the number of bytes written. */
inline size_t emit_json(Tree const& t, EmitOptions const& opts, FILE *f=nullptr)
{
EmitterFile em(opts, f);
return em.emit_as(EMIT_JSON, t, /*error_on_excess*/true).len;
}
/** (2) like (1), but use default emit options */
inline size_t emit_json(Tree const& t, FILE *f=nullptr)
{
EmitterFile em(f);
@@ -181,19 +310,39 @@ inline size_t emit_json(Tree const& t, FILE *f=nullptr)
}
/** emit YAML to the given file. A null file defaults to stdout.
* Return the number of bytes written.
* @overload */
// emit from ConstNodeRef ------------------------
/** (1) emit YAML to the given file. A null file defaults to stdout.
* Return the number of bytes written. */
inline size_t emit_yaml(ConstNodeRef const& r, EmitOptions const& opts, FILE *f=nullptr)
{
if(!detail::is_set_(r))
return {};
EmitterFile em(opts, f);
return em.emit_as(EMIT_YAML, r, /*error_on_excess*/true).len;
}
/** (2) like (1), but use default emit options */
inline size_t emit_yaml(ConstNodeRef const& r, FILE *f=nullptr)
{
if(!detail::is_set_(r))
return {};
EmitterFile em(f);
return em.emit_as(EMIT_YAML, r, /*error_on_excess*/true).len;
}
/** emit JSON to the given file. A null file defaults to stdout.
* Return the number of bytes written.
* @overload */
/** (1) emit JSON to the given file. A null file defaults to stdout.
* Return the number of bytes written. */
inline size_t emit_json(ConstNodeRef const& r, EmitOptions const& opts, FILE *f=nullptr)
{
if(!detail::is_set_(r))
return {};
EmitterFile em(opts, f);
return em.emit_as(EMIT_JSON, r, /*error_on_excess*/true).len;
}
/** (2) like (1), but use default emit options */
inline size_t emit_json(ConstNodeRef const& r, FILE *f=nullptr)
{
if(!detail::is_set_(r))
return {};
EmitterFile em(f);
return em.emit_as(EMIT_JSON, r, /*error_on_excess*/true).len;
}
@@ -208,24 +357,6 @@ inline size_t emit_json(ConstNodeRef const& r, FILE *f=nullptr)
* @{
*/
/** mark a tree or node to be emitted as json when using @ref operator<< . For example:
*
* ```cpp
* Tree t = parse_in_arena("{foo: bar}");
* std::cout << t; // emits YAML
* std::cout << as_json(t); // emits JSON
* ```
*
* @see @ref operator<< */
struct as_json
{
Tree const* tree;
size_t node;
as_json(Tree const& t) : tree(&t), node(t.empty() ? NONE : t.root_id()) {}
as_json(Tree const& t, size_t id) : tree(&t), node(id) {}
as_json(ConstNodeRef const& n) : tree(n.tree()), node(n.id()) {}
};
/** emit YAML to an STL-like ostream */
template<class OStream>
inline OStream& operator<< (OStream& s, Tree const& t)
@@ -240,20 +371,77 @@ inline OStream& operator<< (OStream& s, Tree const& t)
template<class OStream>
inline OStream& operator<< (OStream& s, ConstNodeRef const& n)
{
if(!detail::is_set_(n))
return s;
EmitterOStream<OStream> em(s);
em.emit_as(EMIT_YAML, n);
return s;
}
/** mark a tree or node to be emitted as yaml when using @ref
* operator<<, with options. For example:
*
* ```cpp
* Tree t = parse_in_arena("{foo: bar}");
* std::cout << t; // emits YAML
* std::cout << as_yaml(t); // emits YAML, same as above
* std::cout << as_yaml(t, EmitOptions().max_depth(10)); // emits JSON with a max tree depth
* ```
*
* @see @ref operator<< */
struct as_json
{
Tree const* tree;
size_t node;
EmitOptions options;
as_json(Tree const& t, EmitOptions const& opts={}) : tree(&t), node(t.empty() ? NONE : t.root_id()), options(opts) {}
as_json(Tree const& t, size_t id, EmitOptions const& opts={}) : tree(&t), node(id), options(opts) {}
as_json(ConstNodeRef const& n, EmitOptions const& opts={}) : tree(n.tree()), node(n.id()), options(opts) {}
};
/** mark a tree or node to be emitted as yaml when using @ref
* operator<< . For example:
*
* ```cpp
* Tree t = parse_in_arena("{foo: bar}");
* std::cout << t; // emits YAML
* std::cout << as_json(t); // emits JSON
* std::cout << as_json(t, EmitOptions().max_depth(10)); // emits JSON with a max tree depth
* ```
*
* @see @ref operator<< */
struct as_yaml
{
Tree const* tree;
size_t node;
EmitOptions options;
as_yaml(Tree const& t, EmitOptions const& opts={}) : tree(&t), node(t.empty() ? NONE : t.root_id()), options(opts) {}
as_yaml(Tree const& t, size_t id, EmitOptions const& opts={}) : tree(&t), node(id), options(opts) {}
as_yaml(ConstNodeRef const& n, EmitOptions const& opts={}) : tree(n.tree()), node(n.id()), options(opts) {}
};
/** emit json to an STL-like stream */
template<class OStream>
inline OStream& operator<< (OStream& s, as_json const& j)
{
EmitterOStream<OStream> em(s);
if(!j.tree || j.node == NONE)
return s;
EmitterOStream<OStream> em(j.options, s);
em.emit_as(EMIT_JSON, *j.tree, j.node, true);
return s;
}
/** emit yaml to an STL-like stream */
template<class OStream>
inline OStream& operator<< (OStream& s, as_yaml const& y)
{
if(!y.tree || y.node == NONE)
return s;
EmitterOStream<OStream> em(y.options, s);
em.emit_as(EMIT_YAML, *y.tree, y.node, true);
return s;
}
/** @} */
@@ -264,35 +452,85 @@ inline OStream& operator<< (OStream& s, as_json const& j)
* @{
*/
/** emit YAML to the given buffer. Return a substr trimmed to the emitted YAML.
// emit from tree and node id -----------------------
/** (1) emit YAML to the given buffer. Return a substr trimmed to the emitted YAML.
* @param t the tree to emit.
* @param id the node where to start emitting.
* @param opts emit options.
* @param buf the output buffer.
* @param error_on_excess Raise an error if the space in the buffer is insufficient.
* @overload */
inline substr emit_yaml(Tree const& t, size_t id, substr buf, bool error_on_excess=true)
* @return a substr trimmed to the result in the output buffer. If the buffer is
* insufficient (when error_on_excess is false), the string pointer of the
* result will be set to null, and the length will report the required buffer size. */
inline substr emit_yaml(Tree const& t, id_type id, EmitOptions const& opts, substr buf, bool error_on_excess=true)
{
EmitterBuf em(opts, buf);
return em.emit_as(EMIT_YAML, t, id, error_on_excess);
}
/** (2) like (1), but use default emit options */
inline substr emit_yaml(Tree const& t, id_type id, substr buf, bool error_on_excess=true)
{
EmitterBuf em(buf);
return em.emit_as(EMIT_YAML, t, id, error_on_excess);
}
/** emit JSON to the given buffer. Return a substr trimmed to the emitted JSON.
/** (1) emit JSON to the given buffer. Return a substr trimmed to the emitted JSON.
* @param t the tree to emit.
* @param id the node where to start emitting.
* @param opts emit options.
* @param buf the output buffer.
* @param error_on_excess Raise an error if the space in the buffer is insufficient.
* @overload */
inline substr emit_json(Tree const& t, size_t id, substr buf, bool error_on_excess=true)
* @return a substr trimmed to the result in the output buffer. If the buffer is
* insufficient (when error_on_excess is false), the string pointer of the
* result will be set to null, and the length will report the required buffer size. */
inline substr emit_json(Tree const& t, id_type id, EmitOptions const& opts, substr buf, bool error_on_excess=true)
{
EmitterBuf em(opts, buf);
return em.emit_as(EMIT_JSON, t, id, error_on_excess);
}
/** (2) like (1), but use default emit options */
inline substr emit_json(Tree const& t, id_type id, substr buf, bool error_on_excess=true)
{
EmitterBuf em(buf);
return em.emit_as(EMIT_JSON, t, id, error_on_excess);
}
/** emit YAML to the given buffer. Return a substr trimmed to the emitted YAML.
// emit from root -------------------------
/** (1) emit YAML to the given buffer. Return a substr trimmed to the emitted YAML.
* @param t the tree; will be emitted from the root node.
* @param opts emit options.
* @param buf the output buffer.
* @param error_on_excess Raise an error if the space in the buffer is insufficient.
* @overload */
* @return a substr trimmed to the result in the output buffer. If the buffer is
* insufficient (when error_on_excess is false), the string pointer of the
* result will be set to null, and the length will report the required buffer size. */
inline substr emit_yaml(Tree const& t, EmitOptions const& opts, substr buf, bool error_on_excess=true)
{
EmitterBuf em(opts, buf);
return em.emit_as(EMIT_YAML, t, error_on_excess);
}
/** (2) like (1), but use default emit options */
inline substr emit_yaml(Tree const& t, substr buf, bool error_on_excess=true)
{
EmitterBuf em(buf);
return em.emit_as(EMIT_YAML, t, error_on_excess);
}
/** emit JSON to the given buffer. Return a substr trimmed to the emitted JSON.
/** (1) emit JSON to the given buffer. Return a substr trimmed to the emitted JSON.
* @param t the tree; will be emitted from the root node.
* @param opts emit options.
* @param buf the output buffer.
* @param error_on_excess Raise an error if the space in the buffer is insufficient.
* @overload */
* @return a substr trimmed to the result in the output buffer. If the buffer is
* insufficient (when error_on_excess is false), the string pointer of the
* result will be set to null, and the length will report the required buffer size. */
inline substr emit_json(Tree const& t, EmitOptions const& opts, substr buf, bool error_on_excess=true)
{
EmitterBuf em(opts, buf);
return em.emit_as(EMIT_JSON, t, error_on_excess);
}
/** (2) like (1), but use default emit options */
inline substr emit_json(Tree const& t, substr buf, bool error_on_excess=true)
{
EmitterBuf em(buf);
@@ -300,21 +538,51 @@ inline substr emit_json(Tree const& t, substr buf, bool error_on_excess=true)
}
/** emit YAML to the given buffer. Return a substr trimmed to the emitted YAML.
// emit from ConstNodeRef ------------------------
/** (1) emit YAML to the given buffer. Return a substr trimmed to the emitted YAML.
* @param r the starting node.
* @param buf the output buffer.
* @param opts emit options.
* @param error_on_excess Raise an error if the space in the buffer is insufficient.
* @overload
*/
* @return a substr trimmed to the result in the output buffer. If the buffer is
* insufficient (when error_on_excess is false), the string pointer of the
* result will be set to null, and the length will report the required buffer size. */
inline substr emit_yaml(ConstNodeRef const& r, EmitOptions const& opts, substr buf, bool error_on_excess=true)
{
if(!detail::is_set_(r))
return {};
EmitterBuf em(opts, buf);
return em.emit_as(EMIT_YAML, r, error_on_excess);
}
/** (2) like (1), but use default emit options */
inline substr emit_yaml(ConstNodeRef const& r, substr buf, bool error_on_excess=true)
{
if(!detail::is_set_(r))
return {};
EmitterBuf em(buf);
return em.emit_as(EMIT_YAML, r, error_on_excess);
}
/** emit JSON to the given buffer. Return a substr trimmed to the emitted JSON.
/** (1) emit JSON to the given buffer. Return a substr trimmed to the emitted JSON.
* @param r the starting node.
* @param buf the output buffer.
* @param opts emit options.
* @param error_on_excess Raise an error if the space in the buffer is insufficient.
* @overload
*/
* @return a substr trimmed to the result in the output buffer. If the buffer is
* insufficient (when error_on_excess is false), the string pointer of the
* result will be set to null, and the length will report the required buffer size. */
inline substr emit_json(ConstNodeRef const& r, EmitOptions const& opts, substr buf, bool error_on_excess=true)
{
if(!detail::is_set_(r))
return {};
EmitterBuf em(opts, buf);
return em.emit_as(EMIT_JSON, r, error_on_excess);
}
/** (2) like (1), but use default emit options */
inline substr emit_json(ConstNodeRef const& r, substr buf, bool error_on_excess=true)
{
if(!detail::is_set_(r))
return {};
EmitterBuf em(buf);
return em.emit_as(EMIT_JSON, r, error_on_excess);
}
@@ -322,141 +590,225 @@ inline substr emit_json(ConstNodeRef const& r, substr buf, bool error_on_excess=
//-----------------------------------------------------------------------------
/** emit+resize: emit YAML to the given `std::string`/`std::vector`-like
* container, resizing it as needed to fit the emitted YAML. */
/** @defgroup doc_emit_to_container Emit to resizeable container
*
* @{
*/
// emit from tree and node id ---------------------------
/** (1) emit+resize: emit YAML to the given `std::string`/`std::vector`-like
* container, resizing it as needed to fit the emitted YAML. If @p append is
* set to true, the emitted YAML is appended at the end of the container.
*
* @return a substr trimmed to the emitted YAML (excluding the initial contents, when appending) */
template<class CharOwningContainer>
substr emitrs_yaml(Tree const& t, size_t id, CharOwningContainer * cont)
substr emitrs_yaml(Tree const& t, id_type id, EmitOptions const& opts, CharOwningContainer * cont, bool append=false)
{
substr buf = to_substr(*cont);
substr ret = emit_yaml(t, id, buf, /*error_on_excess*/false);
size_t startpos = append ? cont->size() : 0u;
cont->resize(cont->capacity()); // otherwise the first emit would be certain to fail
substr buf = to_substr(*cont).sub(startpos);
substr ret = emit_yaml(t, id, opts, buf, /*error_on_excess*/false);
if(ret.str == nullptr && ret.len > 0)
{
cont->resize(ret.len);
buf = to_substr(*cont);
ret = emit_yaml(t, id, buf, /*error_on_excess*/true);
cont->resize(startpos + ret.len);
buf = to_substr(*cont).sub(startpos);
ret = emit_yaml(t, id, opts, buf, /*error_on_excess*/true);
}
else
{
cont->resize(startpos + ret.len);
}
return ret;
}
/** emit+resize: emit JSON to the given `std::string`/`std::vector`-like
* container, resizing it as needed to fit the emitted JSON. */
/** (2) like (1), but use default emit options */
template<class CharOwningContainer>
substr emitrs_json(Tree const& t, size_t id, CharOwningContainer * cont)
substr emitrs_yaml(Tree const& t, id_type id, CharOwningContainer * cont, bool append=false)
{
substr buf = to_substr(*cont);
substr ret = emit_json(t, id, buf, /*error_on_excess*/false);
return emitrs_yaml(t, id, EmitOptions{}, cont, append);
}
/** (1) emit+resize: emit JSON to the given `std::string`/`std::vector`-like
* container, resizing it as needed to fit the emitted JSON. If @p append is
* set to true, the emitted YAML is appended at the end of the container.
*
* @return a substr trimmed to the emitted JSON (excluding the initial contents, when appending) */
template<class CharOwningContainer>
substr emitrs_json(Tree const& t, id_type id, EmitOptions const& opts, CharOwningContainer * cont, bool append=false)
{
const size_t startpos = append ? cont->size() : 0u;
cont->resize(cont->capacity()); // otherwise the first emit would be certain to fail
substr buf = to_substr(*cont).sub(startpos);
EmitterBuf em(opts, buf);
substr ret = emit_json(t, id, opts, buf, /*error_on_excess*/false);
if(ret.str == nullptr && ret.len > 0)
{
cont->resize(ret.len);
buf = to_substr(*cont);
ret = emit_json(t, id, buf, /*error_on_excess*/true);
cont->resize(startpos + ret.len);
buf = to_substr(*cont).sub(startpos);
ret = emit_json(t, id, opts, buf, /*error_on_excess*/true);
}
else
{
cont->resize(startpos + ret.len);
}
return ret;
}
/** emit+resize: emit YAML to the given `std::string`/`std::vector`-like
* container, resizing it as needed to fit the emitted YAML. */
/** (2) like (1), but use default emit options */
template<class CharOwningContainer>
CharOwningContainer emitrs_yaml(Tree const& t, size_t id)
substr emitrs_json(Tree const& t, id_type id, CharOwningContainer * cont, bool append=false)
{
return emitrs_json(t, id, EmitOptions{}, cont, append);
}
/** (3) emit+resize: YAML to a newly-created `std::string`/`std::vector`-like container. */
template<class CharOwningContainer>
CharOwningContainer emitrs_yaml(Tree const& t, id_type id, EmitOptions const& opts={})
{
CharOwningContainer c;
emitrs_yaml(t, id, &c);
emitrs_yaml(t, id, opts, &c);
return c;
}
/** emit+resize: emit JSON to the given `std::string`/`std::vector`-like
* container, resizing it as needed to fit the emitted JSON. */
/** (3) emit+resize: JSON to a newly-created `std::string`/`std::vector`-like container. */
template<class CharOwningContainer>
CharOwningContainer emitrs_json(Tree const& t, size_t id)
CharOwningContainer emitrs_json(Tree const& t, id_type id, EmitOptions const& opts={})
{
CharOwningContainer c;
emitrs_json(t, id, &c);
emitrs_json(t, id, opts, &c);
return c;
}
/** emit+resize: YAML to the given `std::string`/`std::vector`-like
* container, resizing it as needed to fit the emitted YAML. */
// emit from root -------------------------
/** (1) emit+resize: YAML to the given `std::string`/`std::vector`-like
* container, resizing it as needed to fit the emitted YAML.
* @return a substr trimmed to the new emitted contents. */
template<class CharOwningContainer>
substr emitrs_yaml(Tree const& t, CharOwningContainer * cont)
substr emitrs_yaml(Tree const& t, EmitOptions const& opts, CharOwningContainer * cont, bool append=false)
{
if(t.empty())
return {};
return emitrs_yaml(t, t.root_id(), cont);
return emitrs_yaml(t, t.root_id(), opts, cont, append);
}
/** emit+resize: JSON to the given `std::string`/`std::vector`-like
* container, resizing it as needed to fit the emitted JSON. */
/** (2) like (1), but use default emit options */
template<class CharOwningContainer>
substr emitrs_json(Tree const& t, CharOwningContainer * cont)
substr emitrs_yaml(Tree const& t, CharOwningContainer * cont, bool append=false)
{
if(t.empty())
return {};
return emitrs_json(t, t.root_id(), cont);
return emitrs_yaml(t, t.root_id(), EmitOptions{}, cont, append);
}
/** (1) emit+resize: JSON to the given `std::string`/`std::vector`-like
* container, resizing it as needed to fit the emitted JSON.
* @return a substr trimmed to the new emitted contents. */
template<class CharOwningContainer>
substr emitrs_json(Tree const& t, EmitOptions const& opts, CharOwningContainer * cont, bool append=false)
{
if(t.empty())
return {};
return emitrs_json(t, t.root_id(), opts, cont, append);
}
/** (2) like (1), but use default emit options */
template<class CharOwningContainer>
substr emitrs_json(Tree const& t, CharOwningContainer * cont, bool append=false)
{
if(t.empty())
return {};
return emitrs_json(t, t.root_id(), EmitOptions{}, cont, append);
}
/** emit+resize: YAML to the given `std::string`/`std::vector`-like container,
* resizing it as needed to fit the emitted YAML. */
/** (3) emit+resize: YAML to a newly-created `std::string`/`std::vector`-like container. */
template<class CharOwningContainer>
CharOwningContainer emitrs_yaml(Tree const& t)
CharOwningContainer emitrs_yaml(Tree const& t, EmitOptions const& opts={})
{
CharOwningContainer c;
if(t.empty())
return c;
emitrs_yaml(t, t.root_id(), &c);
emitrs_yaml(t, t.root_id(), opts, &c);
return c;
}
/** emit+resize: JSON to the given `std::string`/`std::vector`-like container,
* resizing it as needed to fit the emitted JSON. */
/** (3) emit+resize: JSON to a newly-created `std::string`/`std::vector`-like container. */
template<class CharOwningContainer>
CharOwningContainer emitrs_json(Tree const& t)
CharOwningContainer emitrs_json(Tree const& t, EmitOptions const& opts={})
{
CharOwningContainer c;
if(t.empty())
return c;
emitrs_json(t, t.root_id(), &c);
emitrs_json(t, t.root_id(), opts, &c);
return c;
}
/** emit+resize: YAML to the given `std::string`/`std::vector`-like container,
* resizing it as needed to fit the emitted YAML. */
// emit from ConstNodeRef ------------------------
/** (1) emit+resize: YAML to the given `std::string`/`std::vector`-like container,
* resizing it as needed to fit the emitted YAML.
* @return a substr trimmed to the new emitted contents */
template<class CharOwningContainer>
substr emitrs_yaml(ConstNodeRef const& n, CharOwningContainer * cont)
substr emitrs_yaml(ConstNodeRef const& n, EmitOptions const& opts, CharOwningContainer * cont, bool append=false)
{
if(!detail::is_set_(n))
return {};
_RYML_CB_CHECK(n.tree()->callbacks(), n.readable());
return emitrs_yaml(*n.tree(), n.id(), cont);
return emitrs_yaml(*n.tree(), n.id(), opts, cont, append);
}
/** emit+resize: JSON to the given `std::string`/`std::vector`-like container,
* resizing it as needed to fit the emitted JSON. */
/** (2) like (1), but use default emit options */
template<class CharOwningContainer>
substr emitrs_json(ConstNodeRef const& n, CharOwningContainer * cont)
substr emitrs_yaml(ConstNodeRef const& n, CharOwningContainer * cont, bool append=false)
{
if(!detail::is_set_(n))
return {};
_RYML_CB_CHECK(n.tree()->callbacks(), n.readable());
return emitrs_json(*n.tree(), n.id(), cont);
return emitrs_yaml(*n.tree(), n.id(), EmitOptions{}, cont, append);
}
/** (1) emit+resize: JSON to the given `std::string`/`std::vector`-like container,
* resizing it as needed to fit the emitted JSON.
* @return a substr trimmed to the new emitted contents */
template<class CharOwningContainer>
substr emitrs_json(ConstNodeRef const& n, EmitOptions const& opts, CharOwningContainer * cont, bool append=false)
{
if(!detail::is_set_(n))
return {};
_RYML_CB_CHECK(n.tree()->callbacks(), n.readable());
return emitrs_json(*n.tree(), n.id(), opts, cont, append);
}
/** (2) like (1), but use default emit options */
template<class CharOwningContainer>
substr emitrs_json(ConstNodeRef const& n, CharOwningContainer * cont, bool append=false)
{
if(!detail::is_set_(n))
return {};
_RYML_CB_CHECK(n.tree()->callbacks(), n.readable());
return emitrs_json(*n.tree(), n.id(), EmitOptions{}, cont, append);
}
/** emit+resize: YAML to the given `std::string`/`std::vector`-like container,
* resizing it as needed to fit the emitted YAML. */
/** (3) emit+resize: YAML to a newly-created `std::string`/`std::vector`-like container. */
template<class CharOwningContainer>
CharOwningContainer emitrs_yaml(ConstNodeRef const& n)
CharOwningContainer emitrs_yaml(ConstNodeRef const& n, EmitOptions const& opts={})
{
if(!detail::is_set_(n))
return {};
_RYML_CB_CHECK(n.tree()->callbacks(), n.readable());
CharOwningContainer c;
emitrs_yaml(*n.tree(), n.id(), &c);
emitrs_yaml(*n.tree(), n.id(), opts, &c);
return c;
}
/** emit+resize: JSON to the given `std::string`/`std::vector`-like container,
* resizing it as needed to fit the emitted JSON. */
/** (3) emit+resize: JSON to a newly-created `std::string`/`std::vector`-like container. */
template<class CharOwningContainer>
CharOwningContainer emitrs_json(ConstNodeRef const& n)
CharOwningContainer emitrs_json(ConstNodeRef const& n, EmitOptions const& opts={})
{
if(!detail::is_set_(n))
return {};
_RYML_CB_CHECK(n.tree()->callbacks(), n.readable());
CharOwningContainer c;
emitrs_json(*n.tree(), n.id(), &c);
emitrs_json(*n.tree(), n.id(), opts, &c);
return c;
}
/** @} */
@@ -464,7 +816,22 @@ CharOwningContainer emitrs_json(ConstNodeRef const& n)
/** @cond dev */
RYML_DEPRECATE_EMIT inline size_t emit(Tree const& t, size_t id, FILE *f)
#define RYML_DEPRECATE_EMIT \
RYML_DEPRECATED("use emit_yaml() instead. " \
"See https://github.com/biojppm/rapidyaml/issues/120")
#define RYML_DEPRECATE_EMITRS \
RYML_DEPRECATED("use emitrs_yaml() instead. " \
"See https://github.com/biojppm/rapidyaml/issues/120")
// workaround for Qt emit which is a macro;
// see https://github.com/biojppm/rapidyaml/issues/120.
// emit is defined in qobjectdefs.h (as an empty define).
#ifdef emit
#define RYML_TMP_EMIT_
#undef emit
#endif
RYML_DEPRECATE_EMIT inline size_t emit(Tree const& t, id_type id, FILE *f)
{
return emit_yaml(t, id, f);
}
@@ -477,7 +844,7 @@ RYML_DEPRECATE_EMIT inline size_t emit(ConstNodeRef const& r, FILE *f=nullptr)
return emit_yaml(r, f);
}
RYML_DEPRECATE_EMIT inline substr emit(Tree const& t, size_t id, substr buf, bool error_on_excess=true)
RYML_DEPRECATE_EMIT inline substr emit(Tree const& t, id_type id, substr buf, bool error_on_excess=true)
{
return emit_yaml(t, id, buf, error_on_excess);
}
@@ -490,13 +857,18 @@ RYML_DEPRECATE_EMIT inline substr emit(ConstNodeRef const& r, substr buf, bool e
return emit_yaml(r, buf, error_on_excess);
}
#ifdef RYML_TMP_EMIT_
#define emit
#undef RYML_TMP_EMIT_
#endif
template<class CharOwningContainer>
RYML_DEPRECATE_EMITRS substr emitrs(Tree const& t, size_t id, CharOwningContainer * cont)
RYML_DEPRECATE_EMITRS substr emitrs(Tree const& t, id_type id, CharOwningContainer * cont)
{
return emitrs_yaml(t, id, cont);
}
template<class CharOwningContainer>
RYML_DEPRECATE_EMITRS CharOwningContainer emitrs(Tree const& t, size_t id)
RYML_DEPRECATE_EMITRS CharOwningContainer emitrs(Tree const& t, id_type id)
{
return emitrs_yaml<CharOwningContainer>(t, id);
}
@@ -520,15 +892,18 @@ RYML_DEPRECATE_EMITRS CharOwningContainer emitrs(ConstNodeRef const& n)
{
return emitrs_yaml<CharOwningContainer>(n);
}
/** @endcond */
} // namespace yml
} // namespace c4
C4_SUPPRESS_WARNING_GCC_CLANG_POP
#undef RYML_DEPRECATE_EMIT
#undef RYML_DEPRECATE_EMITRS
#include "c4/yml/emit.def.hpp"
#include "c4/yml/emit.def.hpp" // NOLINT
#endif /* _C4_YML_EMIT_HPP_ */

View File

@@ -0,0 +1,194 @@
#ifndef _C4_YML_EVENT_HANDLER_STACK_HPP_
#define _C4_YML_EVENT_HANDLER_STACK_HPP_
#ifndef _C4_YML_DETAIL_STACK_HPP_
#include "c4/yml/detail/stack.hpp"
#endif
#ifndef _C4_YML_NODE_TYPE_HPP_
#include "c4/yml/node_type.hpp"
#endif
#ifndef _C4_YML_DETAIL_PARSER_DBG_HPP_
#include "c4/yml/detail/parser_dbg.hpp"
#endif
#ifndef _C4_YML_PARSER_STATE_HPP_
#include "c4/yml/parser_state.hpp"
#endif
#ifdef RYML_DBG
#ifndef _C4_YML_DETAIL_PRINT_HPP_
#include "c4/yml/detail/print.hpp"
#endif
#endif
// NOLINTBEGIN(hicpp-signed-bitwise)
namespace c4 {
namespace yml {
/** @addtogroup doc_event_handlers
* @{ */
namespace detail {
using pfn_relocate_arena = void (*)(void*, csubstr prev_arena, substr next_arena);
} // detail
/** Use this class a base of implementations of event handler to
* simplify the stack logic. */
template<class HandlerImpl, class HandlerState>
struct EventHandlerStack
{
static_assert(std::is_base_of<ParserState, HandlerState>::value,
"ParserState must be a base of HandlerState");
using state = HandlerState;
using pfn_relocate_arena = detail::pfn_relocate_arena;
public:
detail::stack<state> m_stack;
state *C4_RESTRICT m_curr; ///< current stack level: top of the stack. cached here for easier access.
state *C4_RESTRICT m_parent; ///< parent of the current stack level.
pfn_relocate_arena m_relocate_arena; ///< callback when the arena gets relocated
void * m_relocate_arena_data;
protected:
EventHandlerStack() : m_stack(), m_curr(), m_parent(), m_relocate_arena(), m_relocate_arena_data() {}
EventHandlerStack(Callbacks const& cb) : m_stack(cb), m_curr(), m_parent(), m_relocate_arena(), m_relocate_arena_data() {}
protected:
void _stack_start_parse(const char *filename, pfn_relocate_arena relocate_arena, void *relocate_arena_data)
{
_RYML_CB_ASSERT(m_stack.m_callbacks, m_curr != nullptr);
_RYML_CB_ASSERT(m_stack.m_callbacks, relocate_arena != nullptr);
_RYML_CB_ASSERT(m_stack.m_callbacks, relocate_arena_data != nullptr);
m_curr->start_parse(filename, m_curr->node_id);
m_relocate_arena = relocate_arena;
m_relocate_arena_data = relocate_arena_data;
}
void _stack_finish_parse()
{
}
protected:
void _stack_reset_root()
{
m_stack.clear();
m_stack.push({});
m_parent = nullptr;
m_curr = &m_stack.top();
}
void _stack_reset_non_root()
{
m_stack.clear();
m_stack.push({}); // parent
m_stack.push({}); // node
m_parent = &m_stack.top(1);
m_curr = &m_stack.top();
}
void _stack_push()
{
m_stack.push_top();
m_parent = &m_stack.top(1); // don't use m_curr. watch out for relocations inside the prev push
m_curr = &m_stack.top();
m_curr->reset_after_push();
}
void _stack_pop()
{
_RYML_CB_ASSERT(m_stack.m_callbacks, m_parent);
_RYML_CB_ASSERT(m_stack.m_callbacks, m_stack.size() > 1);
m_parent->reset_before_pop(*m_curr);
m_stack.pop();
m_parent = m_stack.size() > 1 ? &m_stack.top(1) : nullptr;
m_curr = &m_stack.top();
#ifdef RYML_DBG
if(m_parent)
_c4dbgpf("popped! top is now node={} (parent={})", m_curr->node_id, m_parent->node_id);
else
_c4dbgpf("popped! top is now node={} @ ROOT", m_curr->node_id);
#endif
}
protected:
// undefined at the end
#define _has_any_(bits) (static_cast<HandlerImpl const* C4_RESTRICT>(this)->template _has_any__<bits>())
bool _stack_should_push_on_begin_doc() const
{
const bool is_root = (m_stack.size() == 1u);
return is_root && (_has_any_(DOC|VAL|MAP|SEQ) || m_curr->has_children);
}
bool _stack_should_pop_on_end_doc() const
{
const bool is_root = (m_stack.size() == 1u);
return !is_root && _has_any_(DOC);
}
protected:
void _stack_relocate_to_new_arena(csubstr prev, substr curr)
{
for(state &st : m_stack)
{
if(st.line_contents.rem.is_sub(prev))
st.line_contents.rem = _stack_relocate_to_new_arena(st.line_contents.rem, prev, curr);
if(st.line_contents.full.is_sub(prev))
st.line_contents.full = _stack_relocate_to_new_arena(st.line_contents.full, prev, curr);
if(st.line_contents.stripped.is_sub(prev))
st.line_contents.stripped = _stack_relocate_to_new_arena(st.line_contents.stripped, prev, curr);
}
_RYML_CB_ASSERT(m_stack.m_callbacks, m_relocate_arena != nullptr);
_RYML_CB_ASSERT(m_stack.m_callbacks, m_relocate_arena_data != nullptr);
m_relocate_arena(m_relocate_arena_data, prev, curr);
}
substr _stack_relocate_to_new_arena(csubstr s, csubstr prev, substr curr)
{
_RYML_CB_ASSERT(m_stack.m_callbacks, prev.is_super(s));
auto pos = s.str - prev.str;
substr out = {curr.str + pos, s.len};
_RYML_CB_ASSERT(m_stack.m_callbacks, curr.is_super(out));
return out;
}
public:
/** Check whether the current parse tokens are trailing on the
* previous doc, and raise an error if they are. This function is
* called by the parse engine (not the event handler) before a doc
* is started. */
void check_trailing_doc_token() const
{
const bool is_root = (m_stack.size() == 1u);
const bool isndoc = (m_curr->flags & NDOC) != 0;
const bool suspicious = _has_any_(MAP|SEQ|VAL);
_c4dbgpf("target={} isroot={} suspicious={} ndoc={}", m_curr->node_id, is_root, suspicious, isndoc);
if((is_root || _has_any_(DOC)) && suspicious && !isndoc)
_RYML_CB_ERR_(m_stack.m_callbacks, "parse error", m_curr->pos);
}
protected:
#undef _has_any_
};
/** @} */
} // namespace yml
} // namespace c4
// NOLINTEND(hicpp-signed-bitwise)
#endif /* _C4_YML_EVENT_HANDLER_STACK_HPP_ */

View File

@@ -0,0 +1,769 @@
#ifndef _C4_YML_EVENT_HANDLER_TREE_HPP_
#define _C4_YML_EVENT_HANDLER_TREE_HPP_
#ifndef _C4_YML_TREE_HPP_
#include "c4/yml/tree.hpp"
#endif
#ifndef _C4_YML_EVENT_HANDLER_STACK_HPP_
#include "c4/yml/event_handler_stack.hpp"
#endif
C4_SUPPRESS_WARNING_MSVC_WITH_PUSH(4702) // unreachable code
// NOLINTBEGIN(hicpp-signed-bitwise)
namespace c4 {
namespace yml {
/** @addtogroup doc_event_handlers
* @{ */
/** The stack state needed specifically by @ref EventHandlerTree */
struct EventHandlerTreeState : public ParserState
{
NodeData *tr_data;
};
/** The event handler to create a ryml @ref Tree. See the
* documentation for @ref doc_event_handlers, which has important
* notes about the event model used by rapidyaml. */
struct EventHandlerTree : public EventHandlerStack<EventHandlerTree, EventHandlerTreeState>
{
/** @name types
* @{ */
using state = EventHandlerTreeState;
/** @} */
public:
/** @cond dev */
Tree *C4_RESTRICT m_tree;
id_type m_id;
size_t m_num_directives;
bool m_yaml_directive;
#ifdef RYML_DBG
#define _enable_(bits) _enable__<bits>(); _c4dbgpf("node[{}]: enable {}", m_curr->node_id, #bits)
#define _disable_(bits) _disable__<bits>(); _c4dbgpf("node[{}]: disable {}", m_curr->node_id, #bits)
#else
#define _enable_(bits) _enable__<bits>()
#define _disable_(bits) _disable__<bits>()
#endif
#define _has_any_(bits) _has_any__<bits>()
/** @endcond */
public:
/** @name construction and resetting
* @{ */
EventHandlerTree() : EventHandlerStack(), m_tree(), m_id(NONE), m_num_directives(), m_yaml_directive() {}
EventHandlerTree(Callbacks const& cb) : EventHandlerStack(cb), m_tree(), m_id(NONE), m_num_directives(), m_yaml_directive() {}
EventHandlerTree(Tree *tree, id_type id) : EventHandlerStack(tree->callbacks()), m_tree(tree), m_id(id), m_num_directives(), m_yaml_directive()
{
reset(tree, id);
}
void reset(Tree *tree, id_type id)
{
if(C4_UNLIKELY(!tree))
_RYML_CB_ERR(m_stack.m_callbacks, "null tree");
if(C4_UNLIKELY(id >= tree->capacity()))
_RYML_CB_ERR(tree->callbacks(), "invalid node");
if(C4_UNLIKELY(!tree->is_root(id)))
if(C4_UNLIKELY(tree->is_map(tree->parent(id))))
if(C4_UNLIKELY(!tree->has_key(id)))
_RYML_CB_ERR(tree->callbacks(), "destination node belongs to a map and has no key");
m_tree = tree;
m_id = id;
if(m_tree->is_root(id))
{
_stack_reset_root();
_reset_parser_state(m_curr, id, m_tree->root_id());
}
else
{
_stack_reset_non_root();
_reset_parser_state(m_parent, id, m_tree->parent(id));
_reset_parser_state(m_curr, id, id);
}
m_num_directives = 0;
m_yaml_directive = false;
}
/** @} */
public:
/** @name parse events
* @{ */
void start_parse(const char* filename, detail::pfn_relocate_arena relocate_arena, void *relocate_arena_data)
{
_RYML_CB_ASSERT(m_stack.m_callbacks, m_tree != nullptr);
this->_stack_start_parse(filename, relocate_arena, relocate_arena_data);
}
void finish_parse()
{
_RYML_CB_ASSERT(m_stack.m_callbacks, m_tree != nullptr);
if(m_num_directives && !m_tree->is_stream(m_tree->root_id()))
_RYML_CB_ERR_(m_stack.m_callbacks, "directives cannot be used without a document", {});
this->_stack_finish_parse();
/* This pointer is temporary. Remember that:
*
* - this handler object may be held by the user
* - it may be used with a temporary tree inside the parse function
* - when the parse function returns the temporary tree, its address
* will change
*
* As a result, the user could try to read the tree from m_tree, and
* end up reading the stale temporary object.
*
* So it is better to clear it here; then the user will get an obvious
* segfault if reading from m_tree. */
m_tree = nullptr;
}
void cancel_parse()
{
m_tree = nullptr;
}
/** @} */
public:
/** @name YAML stream events */
/** @{ */
C4_ALWAYS_INLINE void begin_stream() const noexcept { /*nothing to do*/ }
C4_ALWAYS_INLINE void end_stream() const noexcept { /*nothing to do*/ }
/** @} */
public:
/** @name YAML document events */
/** @{ */
/** implicit doc start (without ---) */
void begin_doc()
{
_c4dbgp("begin_doc");
if(_stack_should_push_on_begin_doc())
{
_c4dbgp("push!");
_set_root_as_stream();
_push();
_enable_(DOC);
}
}
/** implicit doc end (without ...) */
void end_doc()
{
_c4dbgp("end_doc");
if(_stack_should_pop_on_end_doc())
{
_remove_speculative();
_c4dbgp("pop!");
_pop();
}
}
/** explicit doc start, with --- */
void begin_doc_expl()
{
_c4dbgp("begin_doc_expl");
_RYML_CB_ASSERT(m_stack.m_callbacks, m_tree->root_id() == m_curr->node_id);
if(!m_tree->is_stream(m_tree->root_id())) //if(_should_push_on_begin_doc())
{
_c4dbgp("ensure stream");
_set_root_as_stream();
id_type first = m_tree->first_child(m_tree->root_id());
_RYML_CB_ASSERT(m_stack.m_callbacks, m_tree->is_stream(m_tree->root_id()));
_RYML_CB_ASSERT(m_stack.m_callbacks, m_tree->num_children(m_tree->root_id()) == 1u);
if(m_tree->has_children(first) || m_tree->is_val(first))
{
_c4dbgp("push!");
_push();
}
else
{
_c4dbgp("tweak");
_push();
_remove_speculative();
m_curr->node_id = m_tree->last_child(m_tree->root_id());
m_curr->tr_data = m_tree->_p(m_curr->node_id);
}
}
else
{
_c4dbgp("push!");
_push();
}
_enable_(DOC);
}
/** explicit doc end, with ... */
void end_doc_expl()
{
_c4dbgp("end_doc_expl");
_remove_speculative();
if(_stack_should_pop_on_end_doc())
{
_c4dbgp("pop!");
_pop();
}
m_yaml_directive = false;
}
/** @} */
public:
/** @name YAML map events */
/** @{ */
void begin_map_key_flow()
{
_RYML_CB_ERR_(m_stack.m_callbacks, "ryml trees cannot handle containers as keys", m_curr->pos);
}
void begin_map_key_block()
{
_RYML_CB_ERR_(m_stack.m_callbacks, "ryml trees cannot handle containers as keys", m_curr->pos);
}
void begin_map_val_flow()
{
_c4dbgpf("node[{}]: begin_map_val_flow", m_curr->node_id);
_RYML_CB_CHECK(m_stack.m_callbacks, !_has_any_(VAL));
_enable_(MAP|FLOW_SL);
_save_loc();
_push();
}
void begin_map_val_block()
{
_c4dbgpf("node[{}]: begin_map_val_block", m_curr->node_id);
_RYML_CB_CHECK(m_stack.m_callbacks, !_has_any_(VAL));
_enable_(MAP|BLOCK);
_save_loc();
_push();
}
void end_map()
{
_pop();
_c4dbgpf("node[{}]: end_map_val", m_curr->node_id);
}
/** @} */
public:
/** @name YAML seq events */
/** @{ */
void begin_seq_key_flow()
{
_RYML_CB_ERR_(m_stack.m_callbacks, "ryml trees cannot handle containers as keys", m_curr->pos);
}
void begin_seq_key_block()
{
_RYML_CB_ERR_(m_stack.m_callbacks, "ryml trees cannot handle containers as keys", m_curr->pos);
}
void begin_seq_val_flow()
{
_c4dbgpf("node[{}]: begin_seq_val_flow", m_curr->node_id);
_RYML_CB_CHECK(m_stack.m_callbacks, !_has_any_(VAL));
_enable_(SEQ|FLOW_SL);
_save_loc();
_push();
}
void begin_seq_val_block()
{
_c4dbgpf("node[{}]: begin_seq_val_block", m_curr->node_id);
_RYML_CB_CHECK(m_stack.m_callbacks, !_has_any_(VAL));
_enable_(SEQ|BLOCK);
_save_loc();
_push();
}
void end_seq()
{
_pop();
_c4dbgpf("node[{}]: end_seq_val", m_curr->node_id);
}
/** @} */
public:
/** @name YAML structure events */
/** @{ */
void add_sibling()
{
_RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
_RYML_CB_ASSERT(m_stack.m_callbacks, m_parent);
_RYML_CB_ASSERT(m_stack.m_callbacks, m_tree->has_children(m_parent->node_id));
NodeData const* prev = m_tree->m_buf; // watchout against relocation of the tree nodes
_set_state_(m_curr, m_tree->_append_child__unprotected(m_parent->node_id));
if(prev != m_tree->m_buf)
_refresh_after_relocation();
_c4dbgpf("node[{}]: added sibling={} prev={}", m_parent->node_id, m_curr->node_id, m_tree->prev_sibling(m_curr->node_id));
}
/** set the previous val as the first key of a new map, with flow style.
*
* See the documentation for @ref doc_event_handlers, which has
* important notes about this event.
*/
void actually_val_is_first_key_of_new_map_flow()
{
if(C4_UNLIKELY(m_tree->is_container(m_curr->node_id)))
_RYML_CB_ERR_(m_stack.m_callbacks, "ryml trees cannot handle containers as keys", m_curr->pos);
_RYML_CB_ASSERT(m_stack.m_callbacks, m_parent);
_RYML_CB_ASSERT(m_stack.m_callbacks, m_tree->is_seq(m_parent->node_id));
_RYML_CB_ASSERT(m_stack.m_callbacks, !m_tree->is_container(m_curr->node_id));
_RYML_CB_ASSERT(m_stack.m_callbacks, !m_tree->has_key(m_curr->node_id));
const NodeData tmp = _val2key_(*m_curr->tr_data);
_disable_(_VALMASK|VAL_STYLE);
m_curr->tr_data->m_val = {};
begin_map_val_flow();
m_curr->tr_data->m_type = tmp.m_type;
m_curr->tr_data->m_key = tmp.m_key;
}
/** like its flow counterpart, but this function can only be
* called after the end of a flow-val at root or doc level.
*
* See the documentation for @ref doc_event_handlers, which has
* important notes about this event.
*/
void actually_val_is_first_key_of_new_map_block()
{
_RYML_CB_ERR_(m_stack.m_callbacks, "ryml trees cannot handle containers as keys", m_curr->pos);
}
/** @} */
public:
/** @name YAML scalar events */
/** @{ */
C4_ALWAYS_INLINE void set_key_scalar_plain_empty() noexcept
{
_c4dbgpf("node[{}]: set key scalar plain as empty", m_curr->node_id);
m_curr->tr_data->m_key.scalar = {};
_enable_(KEY|KEY_PLAIN|KEYNIL);
}
C4_ALWAYS_INLINE void set_val_scalar_plain_empty() noexcept
{
_c4dbgpf("node[{}]: set val scalar plain as empty", m_curr->node_id);
m_curr->tr_data->m_val.scalar = {};
_enable_(VAL|VAL_PLAIN|VALNIL);
}
C4_ALWAYS_INLINE void set_key_scalar_plain(csubstr scalar) noexcept
{
_c4dbgpf("node[{}]: set key scalar plain: [{}]~~~{}~~~", m_curr->node_id, scalar.len, scalar);
m_curr->tr_data->m_key.scalar = scalar;
_enable_(KEY|KEY_PLAIN);
}
C4_ALWAYS_INLINE void set_val_scalar_plain(csubstr scalar) noexcept
{
_c4dbgpf("node[{}]: set val scalar plain: [{}]~~~{}~~~", m_curr->node_id, scalar.len, scalar);
m_curr->tr_data->m_val.scalar = scalar;
_enable_(VAL|VAL_PLAIN);
}
C4_ALWAYS_INLINE void set_key_scalar_dquoted(csubstr scalar) noexcept
{
_c4dbgpf("node[{}]: set key scalar dquot: [{}]~~~{}~~~", m_curr->node_id, scalar.len, scalar);
m_curr->tr_data->m_key.scalar = scalar;
_enable_(KEY|KEY_DQUO);
}
C4_ALWAYS_INLINE void set_val_scalar_dquoted(csubstr scalar) noexcept
{
_c4dbgpf("node[{}]: set val scalar dquot: [{}]~~~{}~~~", m_curr->node_id, scalar.len, scalar);
m_curr->tr_data->m_val.scalar = scalar;
_enable_(VAL|VAL_DQUO);
}
C4_ALWAYS_INLINE void set_key_scalar_squoted(csubstr scalar) noexcept
{
_c4dbgpf("node[{}]: set key scalar squot: [{}]~~~{}~~~", m_curr->node_id, scalar.len, scalar);
m_curr->tr_data->m_key.scalar = scalar;
_enable_(KEY|KEY_SQUO);
}
C4_ALWAYS_INLINE void set_val_scalar_squoted(csubstr scalar) noexcept
{
_c4dbgpf("node[{}]: set val scalar squot: [{}]~~~{}~~~", m_curr->node_id, scalar.len, scalar);
m_curr->tr_data->m_val.scalar = scalar;
_enable_(VAL|VAL_SQUO);
}
C4_ALWAYS_INLINE void set_key_scalar_literal(csubstr scalar) noexcept
{
_c4dbgpf("node[{}]: set key scalar literal: [{}]~~~{}~~~", m_curr->node_id, scalar.len, scalar);
m_curr->tr_data->m_key.scalar = scalar;
_enable_(KEY|KEY_LITERAL);
}
C4_ALWAYS_INLINE void set_val_scalar_literal(csubstr scalar) noexcept
{
_c4dbgpf("node[{}]: set val scalar literal: [{}]~~~{}~~~", m_curr->node_id, scalar.len, scalar);
m_curr->tr_data->m_val.scalar = scalar;
_enable_(VAL|VAL_LITERAL);
}
C4_ALWAYS_INLINE void set_key_scalar_folded(csubstr scalar) noexcept
{
_c4dbgpf("node[{}]: set key scalar folded: [{}]~~~{}~~~", m_curr->node_id, scalar.len, scalar);
m_curr->tr_data->m_key.scalar = scalar;
_enable_(KEY|KEY_FOLDED);
}
C4_ALWAYS_INLINE void set_val_scalar_folded(csubstr scalar) noexcept
{
_c4dbgpf("node[{}]: set val scalar folded: [{}]~~~{}~~~", m_curr->node_id, scalar.len, scalar);
m_curr->tr_data->m_val.scalar = scalar;
_enable_(VAL|VAL_FOLDED);
}
C4_ALWAYS_INLINE void mark_key_scalar_unfiltered() noexcept
{
_enable_(KEY_UNFILT);
}
C4_ALWAYS_INLINE void mark_val_scalar_unfiltered() noexcept
{
_enable_(VAL_UNFILT);
}
/** @} */
public:
/** @name YAML anchor/reference events */
/** @{ */
void set_key_anchor(csubstr anchor)
{
_c4dbgpf("node[{}]: set key anchor: [{}]~~~{}~~~", m_curr->node_id, anchor.len, anchor);
_RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
if(C4_UNLIKELY(_has_any_(KEYREF)))
_RYML_CB_ERR_(m_tree->callbacks(), "key cannot have both anchor and ref", m_curr->pos);
_RYML_CB_ASSERT(m_tree->callbacks(), !anchor.begins_with('&'));
_enable_(KEYANCH);
m_curr->tr_data->m_key.anchor = anchor;
}
void set_val_anchor(csubstr anchor)
{
_c4dbgpf("node[{}]: set val anchor: [{}]~~~{}~~~", m_curr->node_id, anchor.len, anchor);
_RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
if(C4_UNLIKELY(_has_any_(VALREF)))
_RYML_CB_ERR_(m_tree->callbacks(), "val cannot have both anchor and ref", m_curr->pos);
_RYML_CB_ASSERT(m_tree->callbacks(), !anchor.begins_with('&'));
_enable_(VALANCH);
m_curr->tr_data->m_val.anchor = anchor;
}
void set_key_ref(csubstr ref)
{
_c4dbgpf("node[{}]: set key ref: [{}]~~~{}~~~", m_curr->node_id, ref.len, ref);
_RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
if(C4_UNLIKELY(_has_any_(KEYANCH)))
_RYML_CB_ERR_(m_tree->callbacks(), "key cannot have both anchor and ref", m_curr->pos);
_RYML_CB_ASSERT(m_tree->callbacks(), ref.begins_with('*'));
_enable_(KEY|KEYREF);
m_curr->tr_data->m_key.anchor = ref.sub(1);
m_curr->tr_data->m_key.scalar = ref;
}
void set_val_ref(csubstr ref)
{
_c4dbgpf("node[{}]: set val ref: [{}]~~~{}~~~", m_curr->node_id, ref.len, ref);
_RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
if(C4_UNLIKELY(_has_any_(VALANCH)))
_RYML_CB_ERR_(m_tree->callbacks(), "val cannot have both anchor and ref", m_curr->pos);
_RYML_CB_ASSERT(m_tree->callbacks(), ref.begins_with('*'));
_enable_(VAL|VALREF);
m_curr->tr_data->m_val.anchor = ref.sub(1);
m_curr->tr_data->m_val.scalar = ref;
}
/** @} */
public:
/** @name YAML tag events */
/** @{ */
void set_key_tag(csubstr tag) noexcept
{
_c4dbgpf("node[{}]: set key tag: [{}]~~~{}~~~", m_curr->node_id, tag.len, tag);
_enable_(KEYTAG);
m_curr->tr_data->m_key.tag = tag;
}
void set_val_tag(csubstr tag) noexcept
{
_c4dbgpf("node[{}]: set val tag: [{}]~~~{}~~~", m_curr->node_id, tag.len, tag);
_enable_(VALTAG);
m_curr->tr_data->m_val.tag = tag;
}
/** @} */
public:
/** @name YAML directive events */
/** @{ */
C4_NO_INLINE void add_directive(csubstr directive)
{
_c4dbgpf("% directive! {}", directive);
_RYML_CB_ASSERT(m_tree->callbacks(), directive.begins_with('%'));
if(directive.begins_with("%TAG"))
{
if(C4_UNLIKELY(!m_tree->add_tag_directive(directive)))
_RYML_CB_ERR_(m_stack.m_callbacks, "failed to add directive", m_curr->pos);
}
else if(directive.begins_with("%YAML"))
{
_c4dbgpf("%YAML directive! ignoring...: {}", directive);
if(C4_UNLIKELY(m_yaml_directive))
_RYML_CB_ERR_(m_stack.m_callbacks, "multiple yaml directives", m_curr->pos);
m_yaml_directive = true;
}
else
{
_c4dbgpf("unknown directive! ignoring... {}", directive);
}
++m_num_directives;
}
/** @} */
public:
/** @name arena functions */
/** @{ */
substr alloc_arena(size_t len)
{
_RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
csubstr prev = m_tree->arena();
substr out = m_tree->alloc_arena(len);
substr curr = m_tree->arena();
if(curr.str != prev.str)
_stack_relocate_to_new_arena(prev, curr);
return out;
}
substr alloc_arena(size_t len, substr *relocated)
{
_RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
csubstr prev = m_tree->arena();
if(!prev.is_super(*relocated))
return alloc_arena(len);
substr out = alloc_arena(len);
substr curr = m_tree->arena();
if(curr.str != prev.str)
*relocated = _stack_relocate_to_new_arena(*relocated, prev, curr);
return out;
}
/** @} */
public:
/** @cond dev */
void _reset_parser_state(state* st, id_type parse_root, id_type node)
{
_RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
_set_state_(st, node);
const NodeType type = m_tree->type(node);
#ifdef RYML_DBG
char flagbuf[80];
_c4dbgpf("resetting state: initial flags={}", detail::_parser_flags_to_str(flagbuf, st->flags));
#endif
if(type == NOTYPE)
{
_c4dbgpf("node[{}] is notype", node);
if(m_tree->is_root(parse_root))
{
_c4dbgpf("node[{}] is root", node);
st->flags |= RUNK|RTOP;
}
else
{
_c4dbgpf("node[{}] is not root. setting USTY", node);
st->flags |= USTY;
}
}
else if(type.is_map())
{
_c4dbgpf("node[{}] is map", node);
st->flags |= RMAP|USTY;
}
else if(type.is_seq())
{
_c4dbgpf("node[{}] is map", node);
st->flags |= RSEQ|USTY;
}
else if(type.has_key())
{
_c4dbgpf("node[{}] has key. setting USTY", node);
st->flags |= USTY;
}
else
{
_RYML_CB_ERR(m_tree->callbacks(), "cannot append to node");
}
if(type.is_doc())
{
_c4dbgpf("node[{}] is doc", node);
st->flags |= RDOC;
}
#ifdef RYML_DBG
_c4dbgpf("resetting state: final flags={}", detail::_parser_flags_to_str(flagbuf, st->flags));
#endif
}
/** push a new parent, add a child to the new parent, and set the
* child as the current node */
void _push()
{
_stack_push();
NodeData const* prev = m_tree->m_buf; // watch out against relocation of the tree nodes
m_curr->node_id = m_tree->_append_child__unprotected(m_parent->node_id);
m_curr->tr_data = m_tree->_p(m_curr->node_id);
if(prev != m_tree->m_buf)
_refresh_after_relocation();
_c4dbgpf("pushed! level={}. top is now node={} (parent={})", m_curr->level, m_curr->node_id, m_parent ? m_parent->node_id : NONE);
}
/** end the current scope */
void _pop()
{
_remove_speculative_with_parent();
_stack_pop();
}
public:
template<type_bits bits> C4_HOT C4_ALWAYS_INLINE void _enable__() noexcept
{
m_curr->tr_data->m_type.type = static_cast<NodeType_e>(m_curr->tr_data->m_type.type | bits);
}
template<type_bits bits> C4_HOT C4_ALWAYS_INLINE void _disable__() noexcept
{
m_curr->tr_data->m_type.type = static_cast<NodeType_e>(m_curr->tr_data->m_type.type & (~bits));
}
template<type_bits bits> C4_HOT C4_ALWAYS_INLINE bool _has_any__() const noexcept
{
return (m_curr->tr_data->m_type.type & bits) != 0;
}
public:
C4_ALWAYS_INLINE void _set_state_(state *C4_RESTRICT s, id_type id) const noexcept
{
s->node_id = id;
s->tr_data = m_tree->_p(id);
}
void _refresh_after_relocation()
{
_c4dbgp("tree: refreshing stack data after tree data relocation");
for(auto &st : m_stack)
st.tr_data = m_tree->_p(st.node_id);
}
void _set_root_as_stream()
{
_c4dbgp("set root as stream");
_RYML_CB_ASSERT(m_tree->callbacks(), m_tree->root_id() == 0u);
_RYML_CB_ASSERT(m_tree->callbacks(), m_curr->node_id == 0u);
const bool hack = !m_tree->has_children(m_curr->node_id) && !m_tree->is_val(m_curr->node_id);
if(hack)
m_tree->_p(m_tree->root_id())->m_type.add(VAL);
m_tree->set_root_as_stream();
_RYML_CB_ASSERT(m_tree->callbacks(), m_tree->is_stream(m_tree->root_id()));
_RYML_CB_ASSERT(m_tree->callbacks(), m_tree->has_children(m_tree->root_id()));
_RYML_CB_ASSERT(m_tree->callbacks(), m_tree->is_doc(m_tree->first_child(m_tree->root_id())));
if(hack)
m_tree->_p(m_tree->first_child(m_tree->root_id()))->m_type.rem(VAL);
_set_state_(m_curr, m_tree->root_id());
}
static NodeData _val2key_(NodeData const& C4_RESTRICT d) noexcept
{
NodeData r = d;
r.m_key = d.m_val;
r.m_val = {};
r.m_type = d.m_type;
static_assert((_VALMASK >> 1u) == _KEYMASK, "required for this function to work");
static_assert((VAL_STYLE >> 1u) == KEY_STYLE, "required for this function to work");
r.m_type.type = ((d.m_type.type & (_VALMASK|VAL_STYLE)) >> 1u);
r.m_type.type = (r.m_type.type & ~(_VALMASK|VAL_STYLE));
r.m_type.type = (r.m_type.type | KEY);
return r;
}
void _remove_speculative()
{
_c4dbgp("remove speculative node");
_RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
_RYML_CB_ASSERT(m_tree->callbacks(), !m_tree->empty());
const id_type last_added = m_tree->size() - 1;
if(m_tree->has_parent(last_added))
if(m_tree->_p(last_added)->m_type == NOTYPE)
m_tree->remove(last_added);
}
void _remove_speculative_with_parent()
{
_RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
_RYML_CB_ASSERT(m_tree->callbacks(), !m_tree->empty());
const id_type last_added = m_tree->size() - 1;
_RYML_CB_ASSERT(m_tree->callbacks(), m_tree->has_parent(last_added));
if(m_tree->_p(last_added)->m_type == NOTYPE)
{
_c4dbgpf("remove speculative node with parent. parent={} node={} parent(node)={}", m_parent->node_id, last_added, m_tree->parent(last_added));
m_tree->remove(last_added);
}
}
C4_ALWAYS_INLINE void _save_loc()
{
_RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
_RYML_CB_ASSERT(m_tree->callbacks(), m_tree->_p(m_curr->node_id)->m_val.scalar.len == 0);
m_tree->_p(m_curr->node_id)->m_val.scalar.str = m_curr->line_contents.rem.str;
}
#undef _enable_
#undef _disable_
#undef _has_any_
/** @endcond */
};
/** @} */
} // namespace yml
} // namespace c4
// NOLINTEND(hicpp-signed-bitwise)
C4_SUPPRESS_WARNING_MSVC_POP
#endif /* _C4_YML_EVENT_HANDLER_TREE_HPP_ */

View File

@@ -0,0 +1,512 @@
#ifndef _C4_YML_FILTER_PROCESSOR_HPP_
#define _C4_YML_FILTER_PROCESSOR_HPP_
#include "c4/yml/common.hpp"
#ifdef RYML_DBG
#include "c4/charconv.hpp"
#include "c4/yml/detail/parser_dbg.hpp"
#endif
namespace c4 {
namespace yml {
/** @defgroup doc_filter_processors Scalar filter processors
*
* These are internal classes used by @ref ParseEngine to parse the
* scalars; normally there is no reason for a user to be manually
* using these classes.
*
* @ingroup doc_parse */
/** @{ */
//-----------------------------------------------------------------------------
/** Filters an input string into a different output string */
struct FilterProcessorSrcDst
{
csubstr src;
substr dst;
size_t rpos; ///< read position
size_t wpos; ///< write position
C4_ALWAYS_INLINE FilterProcessorSrcDst(csubstr src_, substr dst_) noexcept
: src(src_)
, dst(dst_)
, rpos(0)
, wpos(0)
{
RYML_ASSERT(!dst.overlaps(src));
}
C4_ALWAYS_INLINE void setwpos(size_t wpos_) noexcept { wpos = wpos_; }
C4_ALWAYS_INLINE void setpos(size_t rpos_, size_t wpos_) noexcept { rpos = rpos_; wpos = wpos_; }
C4_ALWAYS_INLINE void set_at_end() noexcept { skip(src.len - rpos); }
C4_ALWAYS_INLINE bool has_more_chars() const noexcept { return rpos < src.len; }
C4_ALWAYS_INLINE bool has_more_chars(size_t maxpos) const noexcept { RYML_ASSERT(maxpos <= src.len); return rpos < maxpos; }
C4_ALWAYS_INLINE csubstr rem() const noexcept { return src.sub(rpos); }
C4_ALWAYS_INLINE csubstr sofar() const noexcept { return csubstr(dst.str, wpos <= dst.len ? wpos : dst.len); }
C4_ALWAYS_INLINE FilterResult result() const noexcept
{
FilterResult ret;
ret.str.str = wpos <= dst.len ? dst.str : nullptr;
ret.str.len = wpos;
return ret;
}
C4_ALWAYS_INLINE char curr() const noexcept { RYML_ASSERT(rpos < src.len); return src[rpos]; }
C4_ALWAYS_INLINE char next() const noexcept { return rpos+1 < src.len ? src[rpos+1] : '\0'; }
C4_ALWAYS_INLINE bool skipped_chars() const noexcept { return wpos != rpos; }
C4_ALWAYS_INLINE void skip() noexcept { ++rpos; }
C4_ALWAYS_INLINE void skip(size_t num) noexcept { rpos += num; }
C4_ALWAYS_INLINE void set_at(size_t pos, char c) noexcept // NOLINT(readability-make-member-function-const)
{
RYML_ASSERT(pos < wpos);
dst.str[pos] = c;
}
C4_ALWAYS_INLINE void set(char c) noexcept
{
if(wpos < dst.len)
dst.str[wpos] = c;
++wpos;
}
C4_ALWAYS_INLINE void set(char c, size_t num) noexcept
{
RYML_ASSERT(num > 0);
if(wpos + num <= dst.len)
memset(dst.str + wpos, c, num);
wpos += num;
}
C4_ALWAYS_INLINE void copy() noexcept
{
RYML_ASSERT(rpos < src.len);
if(wpos < dst.len)
dst.str[wpos] = src.str[rpos];
++wpos;
++rpos;
}
C4_ALWAYS_INLINE void copy(size_t num) noexcept
{
RYML_ASSERT(num);
RYML_ASSERT(rpos+num <= src.len);
if(wpos + num <= dst.len)
memcpy(dst.str + wpos, src.str + rpos, num);
wpos += num;
rpos += num;
}
C4_ALWAYS_INLINE void translate_esc(char c) noexcept
{
if(wpos < dst.len)
dst.str[wpos] = c;
++wpos;
rpos += 2;
}
C4_ALWAYS_INLINE void translate_esc_bulk(const char *C4_RESTRICT s, size_t nw, size_t nr) noexcept
{
RYML_ASSERT(nw > 0);
RYML_ASSERT(nr > 0);
RYML_ASSERT(rpos+nr <= src.len);
if(wpos+nw <= dst.len)
memcpy(dst.str + wpos, s, nw);
wpos += nw;
rpos += 1 + nr;
}
C4_ALWAYS_INLINE void translate_esc_extending(const char *C4_RESTRICT s, size_t nw, size_t nr) noexcept
{
translate_esc_bulk(s, nw, nr);
}
};
//-----------------------------------------------------------------------------
// filter in place
// debugging scaffold
/** @cond dev */
#if defined(RYML_DBG) && 0
#define _c4dbgip(...) _c4dbgpf(__VA_ARGS__)
#else
#define _c4dbgip(...)
#endif
/** @endcond */
/** Filters in place. While the result may be larger than the source,
* any extending happens only at the end of the string. Consequently,
* it's impossible for characters to be left unfiltered.
*
* @see FilterProcessorInplaceMidExtending */
struct FilterProcessorInplaceEndExtending
{
substr src; ///< the subject string
size_t wcap; ///< write capacity - the capacity of the subject string's buffer
size_t rpos; ///< read position
size_t wpos; ///< write position
C4_ALWAYS_INLINE FilterProcessorInplaceEndExtending(substr src_, size_t wcap_) noexcept
: src(src_)
, wcap(wcap_)
, rpos(0)
, wpos(0)
{
RYML_ASSERT(wcap >= src.len);
}
C4_ALWAYS_INLINE void setwpos(size_t wpos_) noexcept { wpos = wpos_; }
C4_ALWAYS_INLINE void setpos(size_t rpos_, size_t wpos_) noexcept { rpos = rpos_; wpos = wpos_; }
C4_ALWAYS_INLINE void set_at_end() noexcept { skip(src.len - rpos); }
C4_ALWAYS_INLINE bool has_more_chars() const noexcept { return rpos < src.len; }
C4_ALWAYS_INLINE bool has_more_chars(size_t maxpos) const noexcept { RYML_ASSERT(maxpos <= src.len); return rpos < maxpos; }
C4_ALWAYS_INLINE FilterResult result() const noexcept
{
_c4dbgip("inplace: wpos={} wcap={} small={}", wpos, wcap, wpos > rpos);
FilterResult ret;
ret.str.str = (wpos <= wcap) ? src.str : nullptr;
ret.str.len = wpos;
return ret;
}
C4_ALWAYS_INLINE csubstr sofar() const noexcept { return csubstr(src.str, wpos <= wcap ? wpos : wcap); }
C4_ALWAYS_INLINE csubstr rem() const noexcept { return src.sub(rpos); }
C4_ALWAYS_INLINE char curr() const noexcept { RYML_ASSERT(rpos < src.len); return src[rpos]; }
C4_ALWAYS_INLINE char next() const noexcept { return rpos+1 < src.len ? src[rpos+1] : '\0'; }
C4_ALWAYS_INLINE void skip() noexcept { ++rpos; }
C4_ALWAYS_INLINE void skip(size_t num) noexcept { rpos += num; }
void set_at(size_t pos, char c) noexcept
{
RYML_ASSERT(pos < wpos);
const size_t save = wpos;
wpos = pos;
set(c);
wpos = save;
}
void set(char c) noexcept
{
if(wpos < wcap) // respect write-capacity
src.str[wpos] = c;
++wpos;
}
void set(char c, size_t num) noexcept
{
RYML_ASSERT(num);
if(wpos + num <= wcap) // respect write-capacity
memset(src.str + wpos, c, num);
wpos += num;
}
void copy() noexcept
{
RYML_ASSERT(wpos <= rpos);
RYML_ASSERT(rpos < src.len);
if(wpos < wcap) // respect write-capacity
src.str[wpos] = src.str[rpos];
++rpos;
++wpos;
}
void copy(size_t num) noexcept
{
RYML_ASSERT(num);
RYML_ASSERT(rpos+num <= src.len);
RYML_ASSERT(wpos <= rpos);
if(wpos + num <= wcap) // respect write-capacity
{
if(wpos + num <= rpos) // there is no overlap
memcpy(src.str + wpos, src.str + rpos, num);
else // there is overlap
memmove(src.str + wpos, src.str + rpos, num);
}
rpos += num;
wpos += num;
}
void translate_esc(char c) noexcept
{
RYML_ASSERT(rpos + 2 <= src.len);
RYML_ASSERT(wpos <= rpos);
if(wpos < wcap) // respect write-capacity
src.str[wpos] = c;
rpos += 2; // add 1u to account for the escape character
++wpos;
}
void translate_esc_bulk(const char *C4_RESTRICT s, size_t nw, size_t nr) noexcept
{
RYML_ASSERT(nw > 0);
RYML_ASSERT(nr > 0);
RYML_ASSERT(nw <= nr + 1u);
RYML_ASSERT(rpos+nr <= src.len);
RYML_ASSERT(wpos <= rpos);
const size_t wpos_next = wpos + nw;
const size_t rpos_next = rpos + nr + 1u; // add 1u to account for the escape character
RYML_ASSERT(wpos_next <= rpos_next);
if(wpos_next <= wcap)
memcpy(src.str + wpos, s, nw);
rpos = rpos_next;
wpos = wpos_next;
}
C4_ALWAYS_INLINE void translate_esc_extending(const char *C4_RESTRICT s, size_t nw, size_t nr) noexcept
{
translate_esc_bulk(s, nw, nr);
}
};
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
/** Filters in place. The result may be larger than the source, and
* extending may happen anywhere. As a result some characters may be
* left unfiltered when there is no slack in the buffer and the
* write-position would overlap the read-position. Consequently, it's
* possible for characters to be left unfiltered. In YAML, this
* happens only with double-quoted strings, and only with a small
* number of escape sequences such as `\L` which is substituted by three
* bytes. These escape sequences cause a call to translate_esc_extending()
* which is the only entry point to this unfiltered situation.
*
* @see FilterProcessorInplaceMidExtending */
struct FilterProcessorInplaceMidExtending
{
substr src; ///< the subject string
size_t wcap; ///< write capacity - the capacity of the subject string's buffer
size_t rpos; ///< read position
size_t wpos; ///< write position
size_t maxcap; ///< the max capacity needed for filtering the string. This may be larger than the final string size.
bool unfiltered_chars; ///< number of characters that were not added to wpos from lack of capacity
C4_ALWAYS_INLINE FilterProcessorInplaceMidExtending(substr src_, size_t wcap_) noexcept
: src(src_)
, wcap(wcap_)
, rpos(0)
, wpos(0)
, maxcap(src.len)
, unfiltered_chars(false)
{
RYML_ASSERT(wcap >= src.len);
}
C4_ALWAYS_INLINE void setwpos(size_t wpos_) noexcept { wpos = wpos_; }
C4_ALWAYS_INLINE void setpos(size_t rpos_, size_t wpos_) noexcept { rpos = rpos_; wpos = wpos_; }
C4_ALWAYS_INLINE void set_at_end() noexcept { skip(src.len - rpos); }
C4_ALWAYS_INLINE bool has_more_chars() const noexcept { return rpos < src.len; }
C4_ALWAYS_INLINE bool has_more_chars(size_t maxpos) const noexcept { RYML_ASSERT(maxpos <= src.len); return rpos < maxpos; }
C4_ALWAYS_INLINE FilterResultExtending result() const noexcept
{
_c4dbgip("inplace: wpos={} wcap={} unfiltered={} maxcap={}", this->wpos, this->wcap, this->unfiltered_chars, this->maxcap);
FilterResultExtending ret;
ret.str.str = (wpos <= wcap && !unfiltered_chars) ? src.str : nullptr;
ret.str.len = wpos;
ret.reqlen = maxcap;
return ret;
}
C4_ALWAYS_INLINE csubstr sofar() const noexcept { return csubstr(src.str, wpos <= wcap ? wpos : wcap); }
C4_ALWAYS_INLINE csubstr rem() const noexcept { return src.sub(rpos); }
C4_ALWAYS_INLINE char curr() const noexcept { RYML_ASSERT(rpos < src.len); return src[rpos]; }
C4_ALWAYS_INLINE char next() const noexcept { return rpos+1 < src.len ? src[rpos+1] : '\0'; }
C4_ALWAYS_INLINE void skip() noexcept { ++rpos; }
C4_ALWAYS_INLINE void skip(size_t num) noexcept { rpos += num; }
void set_at(size_t pos, char c) noexcept
{
RYML_ASSERT(pos < wpos);
const size_t save = wpos;
wpos = pos;
set(c);
wpos = save;
}
void set(char c) noexcept
{
if(wpos < wcap) // respect write-capacity
{
if((wpos <= rpos) && !unfiltered_chars)
src.str[wpos] = c;
}
else
{
_c4dbgip("inplace: add unwritten {}->{} maxcap={}->{}!", unfiltered_chars, true, maxcap, (wpos+1u > maxcap ? wpos+1u : maxcap));
unfiltered_chars = true;
}
++wpos;
maxcap = wpos > maxcap ? wpos : maxcap;
}
void set(char c, size_t num) noexcept
{
RYML_ASSERT(num);
if(wpos + num <= wcap) // respect write-capacity
{
if((wpos <= rpos) && !unfiltered_chars)
memset(src.str + wpos, c, num);
}
else
{
_c4dbgip("inplace: add unwritten {}->{} maxcap={}->{}!", unfiltered_chars, true, maxcap, (wpos+num > maxcap ? wpos+num : maxcap));
unfiltered_chars = true;
}
wpos += num;
maxcap = wpos > maxcap ? wpos : maxcap;
}
void copy() noexcept
{
RYML_ASSERT(rpos < src.len);
if(wpos < wcap) // respect write-capacity
{
if((wpos < rpos) && !unfiltered_chars) // write only if wpos is behind rpos
src.str[wpos] = src.str[rpos];
}
else
{
_c4dbgip("inplace: add unwritten {}->{} (wpos={}!=rpos={})={} (wpos={}<wcap={}) maxcap={}->{}!", unfiltered_chars, true, wpos, rpos, wpos!=rpos, wpos, wcap, wpos<wcap, maxcap, (wpos+1u > maxcap ? wpos+1u : maxcap));
unfiltered_chars = true;
}
++rpos;
++wpos;
maxcap = wpos > maxcap ? wpos : maxcap;
}
void copy(size_t num) noexcept
{
RYML_ASSERT(num);
RYML_ASSERT(rpos+num <= src.len);
if(wpos + num <= wcap) // respect write-capacity
{
if((wpos < rpos) && !unfiltered_chars) // write only if wpos is behind rpos
{
if(wpos + num <= rpos) // there is no overlap
memcpy(src.str + wpos, src.str + rpos, num);
else // there is overlap
memmove(src.str + wpos, src.str + rpos, num);
}
}
else
{
_c4dbgip("inplace: add unwritten {}->{} (wpos={}!=rpos={})={} (wpos={}<wcap={}) maxcap={}->{}!", unfiltered_chars, true, wpos, rpos, wpos!=rpos, wpos, wcap, wpos<wcap);
unfiltered_chars = true;
}
rpos += num;
wpos += num;
maxcap = wpos > maxcap ? wpos : maxcap;
}
void translate_esc(char c) noexcept
{
RYML_ASSERT(rpos + 2 <= src.len);
if(wpos < wcap) // respect write-capacity
{
if((wpos <= rpos) && !unfiltered_chars)
src.str[wpos] = c;
}
else
{
_c4dbgip("inplace: add unfiltered {}->{} maxcap={}->{}!", unfiltered_chars, true, maxcap, (wpos+1u > maxcap ? wpos+1u : maxcap));
unfiltered_chars = true;
}
rpos += 2;
++wpos;
maxcap = wpos > maxcap ? wpos : maxcap;
}
C4_NO_INLINE void translate_esc_bulk(const char *C4_RESTRICT s, size_t nw, size_t nr) noexcept
{
RYML_ASSERT(nw > 0);
RYML_ASSERT(nr > 0);
RYML_ASSERT(nr+1u >= nw);
const size_t wpos_next = wpos + nw;
const size_t rpos_next = rpos + nr + 1u; // add 1u to account for the escape character
if(wpos_next <= wcap) // respect write-capacity
{
if((wpos <= rpos) && !unfiltered_chars) // write only if wpos is behind rpos
memcpy(src.str + wpos, s, nw);
}
else
{
_c4dbgip("inplace: add unwritten {}->{} (wpos={}!=rpos={})={} (wpos={}<wcap={}) maxcap={}->{}!", unfiltered_chars, true, wpos, rpos, wpos!=rpos, wpos, wcap, wpos<wcap);
unfiltered_chars = true;
}
rpos = rpos_next;
wpos = wpos_next;
maxcap = wpos > maxcap ? wpos : maxcap;
}
C4_NO_INLINE void translate_esc_extending(const char *C4_RESTRICT s, size_t nw, size_t nr) noexcept
{
RYML_ASSERT(nw > 0);
RYML_ASSERT(nr > 0);
RYML_ASSERT(rpos+nr <= src.len);
const size_t wpos_next = wpos + nw;
const size_t rpos_next = rpos + nr + 1u; // add 1u to account for the escape character
if(wpos_next <= rpos_next) // read and write do not overlap. just do a vanilla copy.
{
if((wpos_next <= wcap) && !unfiltered_chars)
memcpy(src.str + wpos, s, nw);
rpos = rpos_next;
wpos = wpos_next;
maxcap = wpos > maxcap ? wpos : maxcap;
}
else // there is overlap. move the (to-be-read) string to the right.
{
const size_t excess = wpos_next - rpos_next;
RYML_ASSERT(wpos_next > rpos_next);
if(src.len + excess <= wcap) // ensure we do not go past the end
{
RYML_ASSERT(rpos+nr+excess <= src.len);
if(wpos_next <= wcap)
{
if(!unfiltered_chars)
{
memmove(src.str + wpos_next, src.str + rpos_next, src.len - rpos_next);
memcpy(src.str + wpos, s, nw);
}
rpos = wpos_next; // wpos, not rpos
}
else
{
rpos = rpos_next;
//const size_t unw = nw > (nr + 1u) ? nw - (nr + 1u) : 0;
_c4dbgip("inplace: add unfiltered {}->{} maxcap={}->{}!", unfiltered_chars, true);
unfiltered_chars = true;
}
wpos = wpos_next;
// extend the string up to capacity
src.len += excess;
maxcap = wpos > maxcap ? wpos : maxcap;
}
else
{
//const size_t unw = nw > (nr + 1u) ? nw - (nr + 1u) : 0;
RYML_ASSERT(rpos_next <= src.len);
const size_t required_size = wpos_next + (src.len - rpos_next);
_c4dbgip("inplace: add unfiltered {}->{} maxcap={}->{}!", unfiltered_chars, true, maxcap, required_size > maxcap ? required_size : maxcap);
RYML_ASSERT(required_size > wcap);
unfiltered_chars = true;
maxcap = required_size > maxcap ? required_size : maxcap;
wpos = wpos_next;
rpos = rpos_next;
}
}
}
};
#undef _c4dbgip
/** @} */
} // namespace yml
} // namespace c4
#endif /* _C4_YML_FILTER_PROCESSOR_HPP_ */

View File

@@ -0,0 +1,24 @@
#ifndef _C4_YML_FWD_HPP_
#define _C4_YML_FWD_HPP_
/** @file fwd.hpp forward declarations */
namespace c4 {
namespace yml {
struct NodeScalar;
struct NodeInit;
struct NodeData;
struct NodeType;
class NodeRef;
class ConstNodeRef;
class Tree;
struct ReferenceResolver;
template<class EventHandler> class ParseEngine;
struct EventHandlerTree;
using Parser = ParseEngine<EventHandlerTree>;
} // namespace c4
} // namespace yml
#endif /* _C4_YML_FWD_HPP_ */

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,275 @@
#ifndef C4_YML_NODE_TYPE_HPP_
#define C4_YML_NODE_TYPE_HPP_
#ifndef _C4_YML_COMMON_HPP_
#include "c4/yml/common.hpp"
#endif
C4_SUPPRESS_WARNING_MSVC_PUSH
C4_SUPPRESS_WARNING_GCC_CLANG_PUSH
C4_SUPPRESS_WARNING_GCC_CLANG("-Wold-style-cast")
namespace c4 {
namespace yml {
/** @addtogroup doc_node_type
*
* @{
*/
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
/** the integral type necessary to cover all the bits for NodeType_e */
using type_bits = uint32_t;
/** a bit mask for marking node types and styles */
typedef enum : type_bits {
#define __(v) (type_bits(1) << v) // a convenience define, undefined below // NOLINT
NOTYPE = 0, ///< no node type or style is set
KEY = __(0), ///< is member of a map
VAL = __(1), ///< a scalar: has a scalar (ie string) value, possibly empty. must be a leaf node, and cannot be MAP or SEQ
MAP = __(2), ///< a map: a parent of KEYVAL/KEYSEQ/KEYMAP nodes
SEQ = __(3), ///< a seq: a parent of VAL/SEQ/MAP nodes
DOC = __(4), ///< a document
STREAM = __(5)|SEQ, ///< a stream: a seq of docs
KEYREF = __(6), ///< a *reference: the key references an &anchor
VALREF = __(7), ///< a *reference: the val references an &anchor
KEYANCH = __(8), ///< the key has an &anchor
VALANCH = __(9), ///< the val has an &anchor
KEYTAG = __(10), ///< the key has a tag
VALTAG = __(11), ///< the val has a tag
KEYNIL = __(12), ///< the key is null (eg `{ : b}` results in a null key)
VALNIL = __(13), ///< the val is null (eg `{a : }` results in a null val)
_TYMASK = __(14)-1, ///< all the bits up to here
//
// unfiltered flags:
//
KEY_UNFILT = __(14), ///< the key scalar was left unfiltered; the parser was set not to filter. @see ParserOptions
VAL_UNFILT = __(15), ///< the val scalar was left unfiltered; the parser was set not to filter. @see ParserOptions
//
// style flags:
//
FLOW_SL = __(16), ///< mark container with single-line flow style (seqs as '[val1,val2], maps as '{key: val,key2: val2}')
FLOW_ML = __(17), ///< (NOT IMPLEMENTED, work in progress) mark container with multi-line flow style (seqs as '[\n val1,\n val2\n], maps as '{\n key: val,\n key2: val2\n}')
BLOCK = __(18), ///< mark container with block style (seqs as '- val\n', maps as 'key: val')
KEY_LITERAL = __(19), ///< mark key scalar as multiline, block literal |
VAL_LITERAL = __(20), ///< mark val scalar as multiline, block literal |
KEY_FOLDED = __(21), ///< mark key scalar as multiline, block folded >
VAL_FOLDED = __(22), ///< mark val scalar as multiline, block folded >
KEY_SQUO = __(23), ///< mark key scalar as single quoted '
VAL_SQUO = __(24), ///< mark val scalar as single quoted '
KEY_DQUO = __(25), ///< mark key scalar as double quoted "
VAL_DQUO = __(26), ///< mark val scalar as double quoted "
KEY_PLAIN = __(27), ///< mark key scalar as plain scalar (unquoted, even when multiline)
VAL_PLAIN = __(28), ///< mark val scalar as plain scalar (unquoted, even when multiline)
//
// type combination masks:
//
KEYVAL = KEY|VAL,
KEYSEQ = KEY|SEQ,
KEYMAP = KEY|MAP,
DOCMAP = DOC|MAP,
DOCSEQ = DOC|SEQ,
DOCVAL = DOC|VAL,
//
// style combination masks:
//
SCALAR_LITERAL = KEY_LITERAL|VAL_LITERAL,
SCALAR_FOLDED = KEY_FOLDED|VAL_FOLDED,
SCALAR_SQUO = KEY_SQUO|VAL_SQUO,
SCALAR_DQUO = KEY_DQUO|VAL_DQUO,
SCALAR_PLAIN = KEY_PLAIN|VAL_PLAIN,
KEYQUO = KEY_SQUO|KEY_DQUO|KEY_FOLDED|KEY_LITERAL, ///< key style is one of ', ", > or |
VALQUO = VAL_SQUO|VAL_DQUO|VAL_FOLDED|VAL_LITERAL, ///< val style is one of ', ", > or |
KEY_STYLE = KEY_LITERAL|KEY_FOLDED|KEY_SQUO|KEY_DQUO|KEY_PLAIN, ///< mask of all the scalar styles for key (not container styles!)
VAL_STYLE = VAL_LITERAL|VAL_FOLDED|VAL_SQUO|VAL_DQUO|VAL_PLAIN, ///< mask of all the scalar styles for val (not container styles!)
SCALAR_STYLE = KEY_STYLE|VAL_STYLE,
CONTAINER_STYLE_FLOW = FLOW_SL|FLOW_ML,
CONTAINER_STYLE_BLOCK = BLOCK,
CONTAINER_STYLE = FLOW_SL|FLOW_ML|BLOCK,
STYLE = SCALAR_STYLE | CONTAINER_STYLE,
//
// mixed masks
_KEYMASK = KEY | KEYQUO | KEYANCH | KEYREF | KEYTAG,
_VALMASK = VAL | VALQUO | VALANCH | VALREF | VALTAG,
#undef __
} NodeType_e;
constexpr C4_ALWAYS_INLINE C4_CONST NodeType_e operator| (NodeType_e lhs, NodeType_e rhs) noexcept { return (NodeType_e)(((type_bits)lhs) | ((type_bits)rhs)); }
constexpr C4_ALWAYS_INLINE C4_CONST NodeType_e operator& (NodeType_e lhs, NodeType_e rhs) noexcept { return (NodeType_e)(((type_bits)lhs) & ((type_bits)rhs)); }
constexpr C4_ALWAYS_INLINE C4_CONST NodeType_e operator>> (NodeType_e bits, uint32_t n) noexcept { return (NodeType_e)(((type_bits)bits) >> n); }
constexpr C4_ALWAYS_INLINE C4_CONST NodeType_e operator<< (NodeType_e bits, uint32_t n) noexcept { return (NodeType_e)(((type_bits)bits) << n); }
constexpr C4_ALWAYS_INLINE C4_CONST NodeType_e operator~ (NodeType_e bits) noexcept { return (NodeType_e)(~(type_bits)bits); }
C4_ALWAYS_INLINE NodeType_e& operator&= (NodeType_e &subject, NodeType_e bits) noexcept { subject = (NodeType_e)((type_bits)subject & (type_bits)bits); return subject; }
C4_ALWAYS_INLINE NodeType_e& operator|= (NodeType_e &subject, NodeType_e bits) noexcept { subject = (NodeType_e)((type_bits)subject | (type_bits)bits); return subject; }
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
/** wraps a NodeType_e element with some syntactic sugar and predicates */
struct RYML_EXPORT NodeType
{
public:
NodeType_e type;
public:
C4_ALWAYS_INLINE NodeType() noexcept : type(NOTYPE) {}
C4_ALWAYS_INLINE NodeType(NodeType_e t) noexcept : type(t) {}
C4_ALWAYS_INLINE NodeType(type_bits t) noexcept : type((NodeType_e)t) {}
C4_ALWAYS_INLINE bool has_any(NodeType_e t) const noexcept { return (type & t) != 0u; }
C4_ALWAYS_INLINE bool has_all(NodeType_e t) const noexcept { return (type & t) == t; }
C4_ALWAYS_INLINE bool has_none(NodeType_e t) const noexcept { return (type & t) == 0; }
C4_ALWAYS_INLINE void set(NodeType_e t) noexcept { type = t; }
C4_ALWAYS_INLINE void add(NodeType_e t) noexcept { type = (type|t); }
C4_ALWAYS_INLINE void rem(NodeType_e t) noexcept { type = (type & ~t); }
C4_ALWAYS_INLINE void addrem(NodeType_e bits_to_add, NodeType_e bits_to_remove) noexcept { type |= bits_to_add; type &= ~bits_to_remove; }
C4_ALWAYS_INLINE void clear() noexcept { type = NOTYPE; }
public:
C4_ALWAYS_INLINE operator NodeType_e & C4_RESTRICT () noexcept { return type; }
C4_ALWAYS_INLINE operator NodeType_e const& C4_RESTRICT () const noexcept { return type; }
public:
/** @name node type queries
* @{ */
/** return a preset string based on the node type */
C4_ALWAYS_INLINE const char *type_str() const noexcept { return type_str(type); }
/** return a preset string based on the node type */
static const char* type_str(NodeType_e t) noexcept;
/** fill a string with the node type flags. If the string is small, returns {null, len} */
C4_ALWAYS_INLINE csubstr type_str(substr buf) const noexcept { return type_str(buf, type); }
/** fill a string with the node type flags. If the string is small, returns {null, len} */
static csubstr type_str(substr buf, NodeType_e t) noexcept;
public:
/** @name node type queries
* @{ */
C4_ALWAYS_INLINE bool is_notype() const noexcept { return type == NOTYPE; }
C4_ALWAYS_INLINE bool is_stream() const noexcept { return ((type & STREAM) == STREAM) != 0; }
C4_ALWAYS_INLINE bool is_doc() const noexcept { return (type & DOC) != 0; }
C4_ALWAYS_INLINE bool is_container() const noexcept { return (type & (MAP|SEQ|STREAM)) != 0; }
C4_ALWAYS_INLINE bool is_map() const noexcept { return (type & MAP) != 0; }
C4_ALWAYS_INLINE bool is_seq() const noexcept { return (type & SEQ) != 0; }
C4_ALWAYS_INLINE bool has_key() const noexcept { return (type & KEY) != 0; }
C4_ALWAYS_INLINE bool has_val() const noexcept { return (type & VAL) != 0; }
C4_ALWAYS_INLINE bool is_val() const noexcept { return (type & KEYVAL) == VAL; }
C4_ALWAYS_INLINE bool is_keyval() const noexcept { return (type & KEYVAL) == KEYVAL; }
C4_ALWAYS_INLINE bool key_is_null() const noexcept { return (type & KEYNIL) != 0; }
C4_ALWAYS_INLINE bool val_is_null() const noexcept { return (type & VALNIL) != 0; }
C4_ALWAYS_INLINE bool has_key_tag() const noexcept { return (type & KEYTAG) != 0; }
C4_ALWAYS_INLINE bool has_val_tag() const noexcept { return (type & VALTAG) != 0; }
C4_ALWAYS_INLINE bool has_key_anchor() const noexcept { return (type & KEYANCH) != 0; }
C4_ALWAYS_INLINE bool has_val_anchor() const noexcept { return (type & VALANCH) != 0; }
C4_ALWAYS_INLINE bool has_anchor() const noexcept { return (type & (KEYANCH|VALANCH)) != 0; }
C4_ALWAYS_INLINE bool is_key_ref() const noexcept { return (type & KEYREF) != 0; }
C4_ALWAYS_INLINE bool is_val_ref() const noexcept { return (type & VALREF) != 0; }
C4_ALWAYS_INLINE bool is_ref() const noexcept { return (type & (KEYREF|VALREF)) != 0; }
C4_ALWAYS_INLINE bool is_key_unfiltered() const noexcept { return (type & (KEY_UNFILT)) != 0; }
C4_ALWAYS_INLINE bool is_val_unfiltered() const noexcept { return (type & (VAL_UNFILT)) != 0; }
RYML_DEPRECATED("use has_key_anchor()") bool is_key_anchor() const noexcept { return has_key_anchor(); }
RYML_DEPRECATED("use has_val_anchor()") bool is_val_anchor() const noexcept { return has_val_anchor(); }
RYML_DEPRECATED("use has_anchor()") bool is_anchor() const noexcept { return has_anchor(); }
RYML_DEPRECATED("use has_anchor() || is_ref()") bool is_anchor_or_ref() const noexcept { return has_anchor() || is_ref(); }
/** @} */
public:
/** @name container+scalar style queries
* @{ */
C4_ALWAYS_INLINE bool is_container_styled() const noexcept { return (type & (CONTAINER_STYLE)) != 0; }
C4_ALWAYS_INLINE bool is_block() const noexcept { return (type & (BLOCK)) != 0; }
C4_ALWAYS_INLINE bool is_flow_sl() const noexcept { return (type & (FLOW_SL)) != 0; }
C4_ALWAYS_INLINE bool is_flow_ml() const noexcept { return (type & (FLOW_ML)) != 0; }
C4_ALWAYS_INLINE bool is_flow() const noexcept { return (type & (FLOW_ML|FLOW_SL)) != 0; }
C4_ALWAYS_INLINE bool is_key_styled() const noexcept { return (type & (KEY_STYLE)) != 0; }
C4_ALWAYS_INLINE bool is_val_styled() const noexcept { return (type & (VAL_STYLE)) != 0; }
C4_ALWAYS_INLINE bool is_key_literal() const noexcept { return (type & (KEY_LITERAL)) != 0; }
C4_ALWAYS_INLINE bool is_val_literal() const noexcept { return (type & (VAL_LITERAL)) != 0; }
C4_ALWAYS_INLINE bool is_key_folded() const noexcept { return (type & (KEY_FOLDED)) != 0; }
C4_ALWAYS_INLINE bool is_val_folded() const noexcept { return (type & (VAL_FOLDED)) != 0; }
C4_ALWAYS_INLINE bool is_key_squo() const noexcept { return (type & (KEY_SQUO)) != 0; }
C4_ALWAYS_INLINE bool is_val_squo() const noexcept { return (type & (VAL_SQUO)) != 0; }
C4_ALWAYS_INLINE bool is_key_dquo() const noexcept { return (type & (KEY_DQUO)) != 0; }
C4_ALWAYS_INLINE bool is_val_dquo() const noexcept { return (type & (VAL_DQUO)) != 0; }
C4_ALWAYS_INLINE bool is_key_plain() const noexcept { return (type & (KEY_PLAIN)) != 0; }
C4_ALWAYS_INLINE bool is_val_plain() const noexcept { return (type & (VAL_PLAIN)) != 0; }
C4_ALWAYS_INLINE bool is_key_quoted() const noexcept { return (type & KEYQUO) != 0; }
C4_ALWAYS_INLINE bool is_val_quoted() const noexcept { return (type & VALQUO) != 0; }
C4_ALWAYS_INLINE bool is_quoted() const noexcept { return (type & (KEYQUO|VALQUO)) != 0; }
C4_ALWAYS_INLINE void set_container_style(NodeType_e style) noexcept { type = ((style & CONTAINER_STYLE) | (type & ~CONTAINER_STYLE)); }
C4_ALWAYS_INLINE void set_key_style(NodeType_e style) noexcept { type = ((style & KEY_STYLE) | (type & ~KEY_STYLE)); }
C4_ALWAYS_INLINE void set_val_style(NodeType_e style) noexcept { type = ((style & VAL_STYLE) | (type & ~VAL_STYLE)); }
/** @} */
};
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
/** @name scalar style helpers
* @{ */
/** choose a YAML emitting style based on the scalar's contents */
RYML_EXPORT NodeType_e scalar_style_choose(csubstr scalar) noexcept;
/** choose a json style based on the scalar's contents */
RYML_EXPORT NodeType_e scalar_style_json_choose(csubstr scalar) noexcept;
/** query whether a scalar can be encoded using single quotes.
* It may not be possible, notably when there is leading
* whitespace after a newline. */
RYML_EXPORT bool scalar_style_query_squo(csubstr s) noexcept;
/** query whether a scalar can be encoded using plain style (no
* quotes, not a literal/folded block scalar). */
RYML_EXPORT bool scalar_style_query_plain(csubstr s) noexcept;
/** YAML-sense query of nullity. returns true if the scalar points
* to `nullptr` or is otherwise equal to one of the strings
* `"~"`,`"null"`,`"Null"`,`"NULL"` */
RYML_EXPORT inline C4_NO_INLINE bool scalar_is_null(csubstr s) noexcept
{
return s.str == nullptr ||
s == "~" ||
s == "null" ||
s == "Null" ||
s == "NULL";
}
/** @} */
/** @} */
} // namespace yml
} // namespace c4
C4_SUPPRESS_WARNING_MSVC_POP
C4_SUPPRESS_WARNING_GCC_CLANG_POP
#endif /* C4_YML_NODE_TYPE_HPP_ */

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,791 @@
#ifndef _C4_YML_PARSE_ENGINE_HPP_
#define _C4_YML_PARSE_ENGINE_HPP_
#ifndef _C4_YML_DETAIL_PARSER_DBG_HPP_
#include "c4/yml/detail/parser_dbg.hpp"
#endif
#ifndef _C4_YML_PARSER_STATE_HPP_
#include "c4/yml/parser_state.hpp"
#endif
#if defined(_MSC_VER)
# pragma warning(push)
# pragma warning(disable: 4251/*needs to have dll-interface to be used by clients of struct*/)
#endif
// NOLINTBEGIN(hicpp-signed-bitwise)
namespace c4 {
namespace yml {
/** @addtogroup doc_parse
* @{ */
/** @defgroup doc_event_handlers Event Handlers
*
* @brief rapidyaml implements its parsing logic with a two-level
* model, where a @ref ParseEngine object reads through the YAML
* source, and dispatches events to an EventHandler bound to the @ref
* ParseEngine. Because @ref ParseEngine is templated on the event
* handler, the binding uses static polymorphism, without any virtual
* functions. The actual handler object can be changed at run time,
* (but of course needs to be the type of the template parameter).
* This is thus a very efficient architecture, and further enables the
* user to provide his own custom handler if he wishes to bypass the
* rapidyaml @ref Tree.
*
* There are two handlers implemented in this project:
*
* - @ref EventHandlerTree is the handler responsible for creating the
* ryml @ref Tree
*
* - @ref EventHandlerYamlStd is the handler responsible for emitting
* standardized [YAML test suite
* events](https://github.com/yaml/yaml-test-suite), used (only) in
* the CI of this project.
*
*
* ### Event model
*
* The event model used by the parse engine and event handlers follows
* very closely the event model in the [YAML test
* suite](https://github.com/yaml/yaml-test-suite).
*
* Consider for example this YAML,
* ```yaml
* {foo: bar,foo2: bar2}
* ```
* which would produce these events in the test-suite parlance:
* ```
* +STR
* +DOC
* +MAP {}
* =VAL :foo
* =VAL :bar
* =VAL :foo2
* =VAL :bar2
* -MAP
* -DOC
* -STR
* ```
*
* For reference, the @ref ParseEngine object will produce this
* sequence of calls to its bound EventHandler:
* ```cpp
* handler.begin_stream();
* handler.begin_doc();
* handler.begin_map_val_flow();
* handler.set_key_scalar_plain("foo");
* handler.set_val_scalar_plain("bar");
* handler.add_sibling();
* handler.set_key_scalar_plain("foo2");
* handler.set_val_scalar_plain("bar2");
* handler.end_map();
* handler.end_doc();
* handler.end_stream();
* ```
*
* For many other examples of all areas of YAML and how ryml's parse
* model corresponds to the YAML standard model, refer to the [unit
* tests for the parse
* engine](https://github.com/biojppm/rapidyaml/tree/master/test/test_parse_engine.cpp).
*
*
* ### Special events
*
* Most of the parsing events adopted by rapidyaml in its event model
* are fairly obvious, but there are two less-obvious events requiring
* some explanation.
*
* These events exist to make it easier to parse some special YAML
* cases. They are called by the parser when a just-handled
* value/container is actually the first key of a new map:
*
* - `actually_val_is_first_key_of_new_map_flow()` (@ref EventHandlerTree::actually_val_is_first_key_of_new_map_flow() "see implementation in EventHandlerTree" / @ref EventHandlerYamlStd::actually_val_is_first_key_of_new_map_flow() "see implementation in EventHandlerYamlStd")
* - `actually_val_is_first_key_of_new_map_block()` (@ref EventHandlerTree::actually_val_is_first_key_of_new_map_block() "see implementation in EventHandlerTree" / @ref EventHandlerYamlStd::actually_val_is_first_key_of_new_map_block() "see implementation in EventHandlerYamlStd")
*
* For example, consider an implicit map inside a seq: `[a: b, c:
* d]` which is parsed as `[{a: b}, {c: d}]`. The standard event
* sequence for this YAML would be the following:
* ```cpp
* handler.begin_seq_val_flow();
* handler.begin_map_val_flow();
* handler.set_key_scalar_plain("a");
* handler.set_val_scalar_plain("b");
* handler.end_map();
* handler.add_sibling();
* handler.begin_map_val_flow();
* handler.set_key_scalar_plain("c");
* handler.set_val_scalar_plain("d");
* handler.end_map();
* handler.end_seq();
* ```
* The problem with this event sequence is that it forces the
* parser to delay setting the val scalar (in this case "a" and
* "c") until it knows whether the scalar is a key or a val. This
* would require the parser to store the scalar until this
* time. For instance, in the example above, the parser should
* delay setting "a" and "c", because they are in fact keys and
* not vals. Until then, the parser would have to store "a" and
* "c" in its internal state. The downside is that this complexity
* cost would apply even if there is no implicit map -- every val
* in a seq would have to be delayed until one of the
* disambiguating subsequent tokens `,-]:` is found.
* By calling this function, the parser can avoid this complexity,
* by preemptively setting the scalar as a val. Then a call to
* this function will create the map and rearrange the scalar as
* key. Now the cost applies only once: when a seqimap starts. So
* the following (easier and cheaper) event sequence below has the
* same effect as the event sequence above:
* ```cpp
* handler.begin_seq_val_flow();
* handler.set_val_scalar_plain("notmap");
* handler.set_val_scalar_plain("a"); // preemptively set "a" as val!
* handler.actually_as_new_map_key(); // create a map, move the "a" val as the key of the first child of the new map
* handler.set_val_scalar_plain("b"); // now "a" is a key and "b" the val
* handler.end_map();
* handler.set_val_scalar_plain("c"); // "c" also as val!
* handler.actually_as_block_flow(); // likewise
* handler.set_val_scalar_plain("d"); // now "c" is a key and "b" the val
* handler.end_map();
* handler.end_seq();
* ```
* This also applies to container keys (although ryml's tree
* cannot accomodate these): the parser can preemptively set a
* container as a val, and call this event to turn that container
* into a key. For example, consider this yaml:
* ```yaml
* [aa, bb]: [cc, dd]
* # ^ ^ ^
* # | | |
* # (2) (1) (3) <- event sequence
* ```
* The standard event sequence for this YAML would be the
* following:
* ```cpp
* handler.begin_map_val_block(); // (1)
* handler.begin_seq_key_flow(); // (2)
* handler.set_val_scalar_plain("aa");
* handler.add_sibling();
* handler.set_val_scalar_plain("bb");
* handler.end_seq();
* handler.begin_seq_val_flow(); // (3)
* handler.set_val_scalar_plain("cc");
* handler.add_sibling();
* handler.set_val_scalar_plain("dd");
* handler.end_seq();
* handler.end_map();
* ```
* The problem with the sequence above is that, reading from
* left-to-right, the parser can only detect the proper calls at
* (1) and (2) once it reaches (1) in the YAML source. So, the
* parser would have to buffer the entire event sequence starting
* from the beginning until it reaches (1). Using this function,
* the parser can do instead:
* ```cpp
* handler.begin_seq_val_flow(); // (2) -- preemptively as val!
* handler.set_val_scalar_plain("aa");
* handler.add_sibling();
* handler.set_val_scalar_plain("bb");
* handler.end_seq();
* handler.actually_as_new_map_key(); // (1) -- adjust when finding that the prev val was actually a key.
* handler.begin_seq_val_flow(); // (3) -- go on as before
* handler.set_val_scalar_plain("cc");
* handler.add_sibling();
* handler.set_val_scalar_plain("dd");
* handler.end_seq();
* handler.end_map();
* ```
*/
class Tree;
class NodeRef;
class ConstNodeRef;
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
/** Options to give to the parser to control its behavior. */
struct RYML_EXPORT ParserOptions
{
private:
typedef enum : uint32_t {
SCALAR_FILTERING = (1u << 0u),
LOCATIONS = (1u << 1u),
DEFAULTS = SCALAR_FILTERING,
} Flags_e;
uint32_t flags = DEFAULTS;
public:
ParserOptions() = default;
public:
/** @name source location tracking */
/** @{ */
/** enable/disable source location tracking */
ParserOptions& locations(bool enabled) noexcept
{
if(enabled)
flags |= LOCATIONS;
else
flags &= ~LOCATIONS;
return *this;
}
/** query source location tracking status */
C4_ALWAYS_INLINE bool locations() const noexcept { return (flags & LOCATIONS); }
/** @} */
public:
/** @name scalar filtering status (experimental; disable at your discretion) */
/** @{ */
/** enable/disable scalar filtering while parsing */
ParserOptions& scalar_filtering(bool enabled) noexcept
{
if(enabled)
flags |= SCALAR_FILTERING;
else
flags &= ~SCALAR_FILTERING;
return *this;
}
/** query scalar filtering status */
C4_ALWAYS_INLINE bool scalar_filtering() const noexcept { return (flags & SCALAR_FILTERING); }
/** @} */
};
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
/** This is the main driver of parsing logic: it scans the YAML or
* JSON source for tokens, and emits the appropriate sequence of
* parsing events to its event handler. The parse engine itself has no
* special limitations, and *can* accomodate containers as keys; it is the
* event handler may introduce additional constraints.
*
* There are two implemented handlers (see @ref doc_event_handlers,
* which has important notes about the event model):
*
* - @ref EventHandlerTree is the handler responsible for creating the
* ryml @ref Tree
*
* - @ref EventHandlerYamlStd is the handler responsible for emitting
* standardized [YAML test suite
* events](https://github.com/yaml/yaml-test-suite), used (only) in
* the CI of this project. This is not part of the library and is
* not installed.
*/
template<class EventHandler>
class ParseEngine
{
public:
using handler_type = EventHandler;
public:
/** @name construction and assignment */
/** @{ */
ParseEngine(EventHandler *evt_handler, ParserOptions opts={});
~ParseEngine();
ParseEngine(ParseEngine &&) noexcept;
ParseEngine(ParseEngine const&);
ParseEngine& operator=(ParseEngine &&) noexcept;
ParseEngine& operator=(ParseEngine const&);
/** @} */
public:
/** @name modifiers */
/** @{ */
/** Reserve a certain capacity for the parsing stack.
* This should be larger than the expected depth of the parsed
* YAML tree.
*
* The parsing stack is the only (potential) heap memory used
* directly by the parser.
*
* If the requested capacity is below the default
* stack size of 16, the memory is used directly in the parser
* object; otherwise it will be allocated from the heap.
*
* @note this reserves memory only for the parser itself; all the
* allocations for the parsed tree will go through the tree's
* allocator (when different).
*
* @note for maximum efficiency, the tree and the arena can (and
* should) also be reserved. */
void reserve_stack(id_type capacity)
{
m_evt_handler->m_stack.reserve(capacity);
}
/** Reserve a certain capacity for the array used to track node
* locations in the source buffer. */
void reserve_locations(size_t num_source_lines)
{
_resize_locations(num_source_lines);
}
RYML_DEPRECATED("filter arena no longer needed")
void reserve_filter_arena(size_t) {}
/** @} */
public:
/** @name getters */
/** @{ */
/** Get the options used to build this parser object. */
ParserOptions const& options() const { return m_options; }
/** Get the current callbacks in the parser. */
Callbacks const& callbacks() const { RYML_ASSERT(m_evt_handler); return m_evt_handler->m_stack.m_callbacks; }
/** Get the name of the latest file parsed by this object. */
csubstr filename() const { return m_file; }
/** Get the latest YAML buffer parsed by this object. */
csubstr source() const { return m_buf; }
/** Get the encoding of the latest YAML buffer parsed by this object.
* If no encoding was specified, UTF8 is assumed as per the YAML standard. */
Encoding_e encoding() const { return m_encoding != NOBOM ? m_encoding : UTF8; }
id_type stack_capacity() const { RYML_ASSERT(m_evt_handler); return m_evt_handler->m_stack.capacity(); }
size_t locations_capacity() const { return m_newline_offsets_capacity; }
RYML_DEPRECATED("filter arena no longer needed")
size_t filter_arena_capacity() const { return 0u; }
/** @} */
public:
/** @name parse methods */
/** @{ */
/** parse YAML in place, emitting events to the current handler */
void parse_in_place_ev(csubstr filename, substr src);
/** parse JSON in place, emitting events to the current handler */
void parse_json_in_place_ev(csubstr filename, substr src);
/** @} */
public:
/** @name deprecated parse methods
* @{ */
/** @cond dev */
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding function in parse.hpp.") typename std::enable_if<U::is_wtree, void>::type parse_in_place(csubstr filename, substr yaml, Tree *t, size_t node_id);
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding function in parse.hpp.") typename std::enable_if<U::is_wtree, void>::type parse_in_place( substr yaml, Tree *t, size_t node_id);
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding function in parse.hpp.") typename std::enable_if<U::is_wtree, void>::type parse_in_place(csubstr filename, substr yaml, Tree *t );
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding function in parse.hpp.") typename std::enable_if<U::is_wtree, void>::type parse_in_place( substr yaml, Tree *t );
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding function in parse.hpp.") typename std::enable_if<U::is_wtree, void>::type parse_in_place(csubstr filename, substr yaml, NodeRef node );
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding function in parse.hpp.") typename std::enable_if<U::is_wtree, void>::type parse_in_place( substr yaml, NodeRef node );
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding function in parse.hpp.") typename std::enable_if<U::is_wtree, Tree>::type parse_in_place(csubstr filename, substr yaml );
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding function in parse.hpp.") typename std::enable_if<U::is_wtree, Tree>::type parse_in_place( substr yaml );
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding function in parse.hpp.") typename std::enable_if<U::is_wtree, void>::type parse_in_arena(csubstr filename, csubstr yaml, Tree *t, size_t node_id);
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding function in parse.hpp.") typename std::enable_if<U::is_wtree, void>::type parse_in_arena( csubstr yaml, Tree *t, size_t node_id);
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding function in parse.hpp.") typename std::enable_if<U::is_wtree, void>::type parse_in_arena(csubstr filename, csubstr yaml, Tree *t );
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding function in parse.hpp.") typename std::enable_if<U::is_wtree, void>::type parse_in_arena( csubstr yaml, Tree *t );
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding function in parse.hpp.") typename std::enable_if<U::is_wtree, void>::type parse_in_arena(csubstr filename, csubstr yaml, NodeRef node );
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding function in parse.hpp.") typename std::enable_if<U::is_wtree, void>::type parse_in_arena( csubstr yaml, NodeRef node );
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding function in parse.hpp.") typename std::enable_if<U::is_wtree, Tree>::type parse_in_arena(csubstr filename, csubstr yaml );
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding function in parse.hpp.") typename std::enable_if<U::is_wtree, Tree>::type parse_in_arena( csubstr yaml );
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding csubstr version in parse.hpp.") typename std::enable_if<U::is_wtree, void>::type parse_in_arena(csubstr filename, substr yaml, Tree *t, size_t node_id);
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding csubstr version in parse.hpp.") typename std::enable_if<U::is_wtree, void>::type parse_in_arena( substr yaml, Tree *t, size_t node_id);
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding csubstr version in parse.hpp.") typename std::enable_if<U::is_wtree, void>::type parse_in_arena(csubstr filename, substr yaml, Tree *t );
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding csubstr version in parse.hpp.") typename std::enable_if<U::is_wtree, void>::type parse_in_arena( substr yaml, Tree *t );
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding csubstr version in parse.hpp.") typename std::enable_if<U::is_wtree, void>::type parse_in_arena(csubstr filename, substr yaml, NodeRef node );
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding csubstr version in parse.hpp.") typename std::enable_if<U::is_wtree, void>::type parse_in_arena( substr yaml, NodeRef node );
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding csubstr version in parse.hpp.") typename std::enable_if<U::is_wtree, Tree>::type parse_in_arena(csubstr filename, substr yaml );
template<class U=EventHandler> RYML_DEPRECATED("removed, deliberately undefined. use the freestanding csubstr version in parse.hpp.") typename std::enable_if<U::is_wtree, Tree>::type parse_in_arena( substr yaml );
/** @endcond */
/** @} */
public:
/** @name locations */
/** @{ */
/** Get the location of a node of the last tree to be parsed by this parser. */
Location location(Tree const& tree, id_type node_id) const;
/** Get the location of a node of the last tree to be parsed by this parser. */
Location location(ConstNodeRef node) const;
/** Get the string starting at a particular location, to the end
* of the parsed source buffer. */
csubstr location_contents(Location const& loc) const;
/** Given a pointer to a buffer position, get the location.
* @param[in] val must be pointing to somewhere in the source
* buffer that was last parsed by this object. */
Location val_location(const char *val) const;
/** @} */
public:
/** @name scalar filtering */
/** @{*/
/** filter a plain scalar */
FilterResult filter_scalar_plain(csubstr scalar, substr dst, size_t indentation);
/** filter a plain scalar in place */
FilterResult filter_scalar_plain_in_place(substr scalar, size_t cap, size_t indentation);
/** filter a single-quoted scalar */
FilterResult filter_scalar_squoted(csubstr scalar, substr dst);
/** filter a single-quoted scalar in place */
FilterResult filter_scalar_squoted_in_place(substr scalar, size_t cap);
/** filter a double-quoted scalar */
FilterResult filter_scalar_dquoted(csubstr scalar, substr dst);
/** filter a double-quoted scalar in place */
FilterResultExtending filter_scalar_dquoted_in_place(substr scalar, size_t cap);
/** filter a block-literal scalar */
FilterResult filter_scalar_block_literal(csubstr scalar, substr dst, size_t indentation, BlockChomp_e chomp);
/** filter a block-literal scalar in place */
FilterResult filter_scalar_block_literal_in_place(substr scalar, size_t cap, size_t indentation, BlockChomp_e chomp);
/** filter a block-folded scalar */
FilterResult filter_scalar_block_folded(csubstr scalar, substr dst, size_t indentation, BlockChomp_e chomp);
/** filter a block-folded scalar in place */
FilterResult filter_scalar_block_folded_in_place(substr scalar, size_t cap, size_t indentation, BlockChomp_e chomp);
/** @} */
private:
struct ScannedScalar
{
substr scalar;
bool needs_filter;
};
struct ScannedBlock
{
substr scalar;
size_t indentation;
BlockChomp_e chomp;
};
bool _is_doc_begin(csubstr s);
bool _is_doc_end(csubstr s);
bool _scan_scalar_plain_blck(ScannedScalar *C4_RESTRICT sc, size_t indentation);
bool _scan_scalar_plain_seq_flow(ScannedScalar *C4_RESTRICT sc);
bool _scan_scalar_plain_seq_blck(ScannedScalar *C4_RESTRICT sc);
bool _scan_scalar_plain_map_flow(ScannedScalar *C4_RESTRICT sc);
bool _scan_scalar_plain_map_blck(ScannedScalar *C4_RESTRICT sc);
bool _scan_scalar_map_json(ScannedScalar *C4_RESTRICT sc);
bool _scan_scalar_seq_json(ScannedScalar *C4_RESTRICT sc);
bool _scan_scalar_plain_unk(ScannedScalar *C4_RESTRICT sc);
bool _is_valid_start_scalar_plain_flow(csubstr s);
ScannedScalar _scan_scalar_squot();
ScannedScalar _scan_scalar_dquot();
void _scan_block(ScannedBlock *C4_RESTRICT sb, size_t indref);
csubstr _scan_anchor();
csubstr _scan_ref_seq();
csubstr _scan_ref_map();
csubstr _scan_tag();
public: // exposed for testing
/** @cond dev */
csubstr _filter_scalar_plain(substr s, size_t indentation);
csubstr _filter_scalar_squot(substr s);
csubstr _filter_scalar_dquot(substr s);
csubstr _filter_scalar_literal(substr s, size_t indentation, BlockChomp_e chomp);
csubstr _filter_scalar_folded(substr s, size_t indentation, BlockChomp_e chomp);
csubstr _maybe_filter_key_scalar_plain(ScannedScalar const& sc, size_t indendation);
csubstr _maybe_filter_val_scalar_plain(ScannedScalar const& sc, size_t indendation);
csubstr _maybe_filter_key_scalar_squot(ScannedScalar const& sc);
csubstr _maybe_filter_val_scalar_squot(ScannedScalar const& sc);
csubstr _maybe_filter_key_scalar_dquot(ScannedScalar const& sc);
csubstr _maybe_filter_val_scalar_dquot(ScannedScalar const& sc);
csubstr _maybe_filter_key_scalar_literal(ScannedBlock const& sb);
csubstr _maybe_filter_val_scalar_literal(ScannedBlock const& sb);
csubstr _maybe_filter_key_scalar_folded(ScannedBlock const& sb);
csubstr _maybe_filter_val_scalar_folded(ScannedBlock const& sb);
/** @endcond */
private:
void _handle_map_block();
void _handle_seq_block();
void _handle_map_flow();
void _handle_seq_flow();
void _handle_seq_imap();
void _handle_map_json();
void _handle_seq_json();
void _handle_unk();
void _handle_unk_json();
void _handle_usty();
void _handle_flow_skip_whitespace();
void _end_map_blck();
void _end_seq_blck();
void _end2_map();
void _end2_seq();
void _begin2_doc();
void _begin2_doc_expl();
void _end2_doc();
void _end2_doc_expl();
void _maybe_begin_doc();
void _maybe_end_doc();
void _start_doc_suddenly();
void _end_doc_suddenly();
void _end_doc_suddenly__pop();
void _end_stream();
void _set_indentation(size_t indentation);
void _save_indentation();
void _handle_indentation_pop_from_block_seq();
void _handle_indentation_pop_from_block_map();
void _handle_indentation_pop(ParserState const* dst);
void _maybe_skip_comment();
void _skip_comment();
void _maybe_skip_whitespace_tokens();
void _maybe_skipchars(char c);
#ifdef RYML_NO_COVERAGE__TO_BE_DELETED
void _maybe_skipchars_up_to(char c, size_t max_to_skip);
#endif
template<size_t N>
void _skipchars(const char (&chars)[N]);
bool _maybe_scan_following_colon() noexcept;
bool _maybe_scan_following_comma() noexcept;
public:
/** @cond dev */
template<class FilterProcessor> auto _filter_plain(FilterProcessor &C4_RESTRICT proc, size_t indentation) -> decltype(proc.result());
template<class FilterProcessor> auto _filter_squoted(FilterProcessor &C4_RESTRICT proc) -> decltype(proc.result());
template<class FilterProcessor> auto _filter_dquoted(FilterProcessor &C4_RESTRICT proc) -> decltype(proc.result());
template<class FilterProcessor> auto _filter_block_literal(FilterProcessor &C4_RESTRICT proc, size_t indentation, BlockChomp_e chomp) -> decltype(proc.result());
template<class FilterProcessor> auto _filter_block_folded(FilterProcessor &C4_RESTRICT proc, size_t indentation, BlockChomp_e chomp) -> decltype(proc.result());
/** @endcond */
public:
/** @cond dev */
template<class FilterProcessor> void _filter_nl_plain(FilterProcessor &C4_RESTRICT proc, size_t indentation);
template<class FilterProcessor> void _filter_nl_squoted(FilterProcessor &C4_RESTRICT proc);
template<class FilterProcessor> void _filter_nl_dquoted(FilterProcessor &C4_RESTRICT proc);
template<class FilterProcessor> bool _filter_ws_handle_to_first_non_space(FilterProcessor &C4_RESTRICT proc);
template<class FilterProcessor> void _filter_ws_copy_trailing(FilterProcessor &C4_RESTRICT proc);
template<class FilterProcessor> void _filter_ws_skip_trailing(FilterProcessor &C4_RESTRICT proc);
template<class FilterProcessor> void _filter_dquoted_backslash(FilterProcessor &C4_RESTRICT proc);
template<class FilterProcessor> void _filter_chomp(FilterProcessor &C4_RESTRICT proc, BlockChomp_e chomp, size_t indentation);
template<class FilterProcessor> size_t _handle_all_whitespace(FilterProcessor &C4_RESTRICT proc, BlockChomp_e chomp);
template<class FilterProcessor> size_t _extend_to_chomp(FilterProcessor &C4_RESTRICT proc, size_t contents_len);
template<class FilterProcessor> void _filter_block_indentation(FilterProcessor &C4_RESTRICT proc, size_t indentation);
template<class FilterProcessor> void _filter_block_folded_newlines(FilterProcessor &C4_RESTRICT proc, size_t indentation, size_t len);
template<class FilterProcessor> size_t _filter_block_folded_newlines_compress(FilterProcessor &C4_RESTRICT proc, size_t num_newl, size_t wpos_at_first_newl);
template<class FilterProcessor> void _filter_block_folded_newlines_leading(FilterProcessor &C4_RESTRICT proc, size_t indentation, size_t len);
template<class FilterProcessor> void _filter_block_folded_indented_block(FilterProcessor &C4_RESTRICT proc, size_t indentation, size_t len, size_t curr_indentation) noexcept;
/** @endcond */
private:
void _line_progressed(size_t ahead);
void _line_ended();
void _line_ended_undo();
bool _finished_file() const;
bool _finished_line() const;
void _scan_line();
substr _peek_next_line(size_t pos=npos) const;
bool _at_line_begin() const
{
return m_evt_handler->m_curr->line_contents.rem.begin() == m_evt_handler->m_curr->line_contents.full.begin();
}
void _relocate_arena(csubstr prev_arena, substr next_arena);
static void _s_relocate_arena(void*, csubstr prev_arena, substr next_arena);
private:
C4_ALWAYS_INLINE bool has_all(ParserFlag_t f) const noexcept { return (m_evt_handler->m_curr->flags & f) == f; }
C4_ALWAYS_INLINE bool has_any(ParserFlag_t f) const noexcept { return (m_evt_handler->m_curr->flags & f) != 0; }
C4_ALWAYS_INLINE bool has_none(ParserFlag_t f) const noexcept { return (m_evt_handler->m_curr->flags & f) == 0; }
static C4_ALWAYS_INLINE bool has_all(ParserFlag_t f, ParserState const* C4_RESTRICT s) noexcept { return (s->flags & f) == f; }
static C4_ALWAYS_INLINE bool has_any(ParserFlag_t f, ParserState const* C4_RESTRICT s) noexcept { return (s->flags & f) != 0; }
static C4_ALWAYS_INLINE bool has_none(ParserFlag_t f, ParserState const* C4_RESTRICT s) noexcept { return (s->flags & f) == 0; }
#ifndef RYML_DBG
C4_ALWAYS_INLINE static void add_flags(ParserFlag_t on, ParserState *C4_RESTRICT s) noexcept { s->flags |= on; }
C4_ALWAYS_INLINE static void addrem_flags(ParserFlag_t on, ParserFlag_t off, ParserState *C4_RESTRICT s) noexcept { s->flags &= ~off; s->flags |= on; }
C4_ALWAYS_INLINE static void rem_flags(ParserFlag_t off, ParserState *C4_RESTRICT s) noexcept { s->flags &= ~off; }
C4_ALWAYS_INLINE void add_flags(ParserFlag_t on) noexcept { m_evt_handler->m_curr->flags |= on; }
C4_ALWAYS_INLINE void addrem_flags(ParserFlag_t on, ParserFlag_t off) noexcept { m_evt_handler->m_curr->flags &= ~off; m_evt_handler->m_curr->flags |= on; }
C4_ALWAYS_INLINE void rem_flags(ParserFlag_t off) noexcept { m_evt_handler->m_curr->flags &= ~off; }
#else
static void add_flags(ParserFlag_t on, ParserState *C4_RESTRICT s);
static void addrem_flags(ParserFlag_t on, ParserFlag_t off, ParserState *C4_RESTRICT s);
static void rem_flags(ParserFlag_t off, ParserState *C4_RESTRICT s);
C4_ALWAYS_INLINE void add_flags(ParserFlag_t on) noexcept { add_flags(on, m_evt_handler->m_curr); }
C4_ALWAYS_INLINE void addrem_flags(ParserFlag_t on, ParserFlag_t off) noexcept { addrem_flags(on, off, m_evt_handler->m_curr); }
C4_ALWAYS_INLINE void rem_flags(ParserFlag_t off) noexcept { rem_flags(off, m_evt_handler->m_curr); }
#endif
private:
void _prepare_locations();
void _resize_locations(size_t sz);
bool _locations_dirty() const;
bool _location_from_cont(Tree const& tree, id_type node, Location *C4_RESTRICT loc) const;
bool _location_from_node(Tree const& tree, id_type node, Location *C4_RESTRICT loc, id_type level) const;
private:
void _reset();
void _free();
void _clr();
#ifdef RYML_DBG
template<class ...Args> void _dbg(csubstr fmt, Args const& C4_RESTRICT ...args) const;
#endif
template<class ...Args> void _err(csubstr fmt, Args const& C4_RESTRICT ...args) const;
template<class ...Args> void _errloc(csubstr fmt, Location const& loc, Args const& C4_RESTRICT ...args) const;
template<class DumpFn> void _fmt_msg(DumpFn &&dumpfn) const;
private:
/** store pending tag or anchor/ref annotations */
struct Annotation
{
struct Entry
{
csubstr str;
size_t indentation;
size_t line;
};
Entry annotations[2];
size_t num_entries;
};
void _handle_colon();
void _add_annotation(Annotation *C4_RESTRICT dst, csubstr str, size_t indentation, size_t line);
void _clear_annotations(Annotation *C4_RESTRICT dst);
bool _has_pending_annotations() const { return m_pending_tags.num_entries || m_pending_anchors.num_entries; }
#ifdef RYML_NO_COVERAGE__TO_BE_DELETED
bool _handle_indentation_from_annotations();
#endif
bool _annotations_require_key_container() const;
void _handle_annotations_before_blck_key_scalar();
void _handle_annotations_before_blck_val_scalar();
void _handle_annotations_before_start_mapblck(size_t current_line);
void _handle_annotations_before_start_mapblck_as_key();
void _handle_annotations_and_indentation_after_start_mapblck(size_t key_indentation, size_t key_line);
size_t _select_indentation_from_annotations(size_t val_indentation, size_t val_line);
void _handle_directive(csubstr rem);
bool _handle_bom();
void _handle_bom(Encoding_e enc);
void _check_tag(csubstr tag);
private:
ParserOptions m_options;
csubstr m_file;
substr m_buf;
public:
/** @cond dev */
EventHandler *C4_RESTRICT m_evt_handler; // NOLINT
/** @endcond */
private:
Annotation m_pending_anchors;
Annotation m_pending_tags;
bool m_was_inside_qmrk;
bool m_doc_empty = true;
size_t m_prev_colon = npos;
Encoding_e m_encoding = UTF8;
private:
size_t *m_newline_offsets;
size_t m_newline_offsets_size;
size_t m_newline_offsets_capacity;
csubstr m_newline_offsets_buf;
};
/** @cond dev */
RYML_EXPORT C4_NO_INLINE size_t _find_last_newline_and_larger_indentation(csubstr s, size_t indentation) noexcept;
/** @endcond */
/** Quickly inspect the source to estimate the number of nodes the
* resulting tree is likely have. If a tree is empty before
* parsing, considerable time will be spent growing it, so calling
* this to reserve the tree size prior to parsing is likely to
* result in a time gain. We encourage using this method before
* parsing, but as always measure its impact in performance to
* obtain a good trade-off.
*
* @note since this method is meant for optimizing performance, it
* is approximate. The result may be actually smaller than the
* resulting number of nodes, notably if the YAML uses implicit
* maps as flow seq members as in `[these: are, individual:
* maps]`. */
RYML_EXPORT id_type estimate_tree_capacity(csubstr src); // NOLINT(readability-redundant-declaration)
/** @} */
} // namespace yml
} // namespace c4
// NOLINTEND(hicpp-signed-bitwise)
#if defined(_MSC_VER)
# pragma warning(pop)
#endif
#endif /* _C4_YML_PARSE_ENGINE_HPP_ */

View File

@@ -0,0 +1,212 @@
#ifndef _C4_YML_PARSER_STATE_HPP_
#define _C4_YML_PARSER_STATE_HPP_
#ifndef _C4_YML_COMMON_HPP_
#include "c4/yml/common.hpp"
#endif
// NOLINTBEGIN(hicpp-signed-bitwise)
namespace c4 {
namespace yml {
/** data type for @ref ParserState_e */
using ParserFlag_t = int;
/** Enumeration of the state flags for the parser */
typedef enum : ParserFlag_t {
RTOP = 0x01 << 0, ///< reading at top level
RUNK = 0x01 << 1, ///< reading unknown state (when starting): must determine whether scalar, map or seq
RMAP = 0x01 << 2, ///< reading a map
RSEQ = 0x01 << 3, ///< reading a seq
FLOW = 0x01 << 4, ///< reading is inside explicit flow chars: [] or {}
BLCK = 0x01 << 5, ///< reading in block mode
QMRK = 0x01 << 6, ///< reading an explicit key (`? key`)
RKEY = 0x01 << 7, ///< reading a scalar as key
RVAL = 0x01 << 9, ///< reading a scalar as val
RKCL = 0x01 << 8, ///< reading the key colon (ie the : after the key in the map)
RNXT = 0x01 << 10, ///< read next val or keyval
SSCL = 0x01 << 11, ///< there's a stored scalar
QSCL = 0x01 << 12, ///< stored scalar was quoted
RSET = 0x01 << 13, ///< the (implicit) map being read is a !!set. @see https://yaml.org/type/set.html
RDOC = 0x01 << 14, ///< reading a document
NDOC = 0x01 << 15, ///< no document mode. a document has ended and another has not started yet.
USTY = 0x01 << 16, ///< reading in unknown style mode - must determine FLOW or BLCK
//! reading an implicit map nested in an explicit seq.
//! eg, {key: [key2: value2, key3: value3]}
//! is parsed as {key: [{key2: value2}, {key3: value3}]}
RSEQIMAP = 0x01 << 17,
} ParserState_e;
#ifdef RYML_DBG
/** @cond dev */
namespace detail {
csubstr _parser_flags_to_str(substr buf, ParserFlag_t flags);
} // namespace
/** @endcond */
#endif
/** Helper to control the line contents while parsing a buffer */
struct LineContents
{
substr rem; ///< the stripped line remainder; initially starts at the first non-space character
size_t indentation; ///< the number of spaces on the beginning of the line
substr full; ///< the full line, including newlines on the right
substr stripped; ///< the stripped line, excluding newlines on the right
LineContents() = default;
void reset_with_next_line(substr buf, size_t offset)
{
RYML_ASSERT(offset <= buf.len);
size_t e = offset;
// get the current line stripped of newline chars
while(e < buf.len && (buf.str[e] != '\n' && buf.str[e] != '\r'))
++e;
RYML_ASSERT(e >= offset);
const substr stripped_ = buf.range(offset, e);
#if defined(__GNUC__) && __GNUC__ == 11
C4_DONT_OPTIMIZE(stripped_);
#endif
// advance pos to include the first line ending
if(e < buf.len && buf.str[e] == '\r')
++e;
if(e < buf.len && buf.str[e] == '\n')
++e;
const substr full_ = buf.range(offset, e);
reset(full_, stripped_);
}
void reset(substr full_, substr stripped_)
{
rem = stripped_;
indentation = stripped_.first_not_of(' '); // find the first column where the character is not a space
full = full_;
stripped = stripped_;
}
C4_ALWAYS_INLINE size_t current_col() const RYML_NOEXCEPT
{
// WARNING: gcc x86 release builds were wrong (eg returning 0
// when the result should be 4 ) when this function was like
// this:
//
//return current_col(rem);
//
// (see below for the full definition of the called overload
// of current_col())
//
// ... so we explicitly inline the code in here:
RYML_ASSERT(rem.str >= full.str);
size_t col = static_cast<size_t>(rem.str - full.str);
return col;
//
// this was happening only on builds specifically with (gcc
// AND x86 AND release); no other builds were having the
// problem: not in debug, not in x64, not in other
// architectures, not in clang, not in visual studio. WTF!?
//
// Enabling debug prints with RYML_DBG made the problem go
// away, so these could not be used to debug the
// problem. Adding prints inside the called current_col() also
// made the problem go away! WTF!???
//
// a prize will be offered to anybody able to explain why this
// was happening.
}
C4_ALWAYS_INLINE size_t current_col(csubstr s) const RYML_NOEXCEPT
{
RYML_ASSERT(s.str >= full.str);
RYML_ASSERT(full.is_super(s));
size_t col = static_cast<size_t>(s.str - full.str);
return col;
}
};
static_assert(std::is_standard_layout<LineContents>::value, "LineContents not standard");
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
struct ParserState
{
LineContents line_contents;
Location pos;
ParserFlag_t flags;
size_t indref; ///< the reference indentation in the current block scope
id_type level;
id_type node_id; ///< don't hold a pointer to the node as it will be relocated during tree resizes
size_t scalar_col; // the column where the scalar (or its quotes) begin
bool more_indented;
bool has_children;
ParserState() = default;
void start_parse(const char *file, id_type node_id_)
{
level = 0;
pos.name = to_csubstr(file);
pos.offset = 0;
pos.line = 1;
pos.col = 1;
node_id = node_id_;
more_indented = false;
scalar_col = 0;
indref = 0;
has_children = false;
}
void reset_after_push()
{
node_id = NONE;
indref = npos;
more_indented = false;
++level;
has_children = false;
}
C4_ALWAYS_INLINE void reset_before_pop(ParserState const& to_pop)
{
pos = to_pop.pos;
line_contents = to_pop.line_contents;
}
public:
C4_ALWAYS_INLINE bool at_line_beginning() const noexcept
{
return line_contents.rem.str == line_contents.full.str;
}
C4_ALWAYS_INLINE bool indentation_eq() const noexcept
{
RYML_ASSERT(indref != npos);
return line_contents.indentation != npos && line_contents.indentation == indref;
}
C4_ALWAYS_INLINE bool indentation_ge() const noexcept
{
RYML_ASSERT(indref != npos);
return line_contents.indentation != npos && line_contents.indentation >= indref;
}
C4_ALWAYS_INLINE bool indentation_gt() const noexcept
{
RYML_ASSERT(indref != npos);
return line_contents.indentation != npos && line_contents.indentation > indref;
}
C4_ALWAYS_INLINE bool indentation_lt() const noexcept
{
RYML_ASSERT(indref != npos);
return line_contents.indentation != npos && line_contents.indentation < indref;
}
};
static_assert(std::is_standard_layout<ParserState>::value, "ParserState not standard");
} // namespace yml
} // namespace c4
// NOLINTEND(hicpp-signed-bitwise)
#endif /* _C4_YML_PARSER_STATE_HPP_ */

View File

@@ -0,0 +1,88 @@
#ifndef _C4_YML_REFERENCE_RESOLVER_HPP_
#define _C4_YML_REFERENCE_RESOLVER_HPP_
#include "c4/yml/tree.hpp"
#include "c4/yml/detail/stack.hpp"
namespace c4 {
namespace yml {
/** @addtogroup doc_ref_utils
* @{
*/
/** Reusable object to resolve references/aliases in a @ref Tree. */
struct RYML_EXPORT ReferenceResolver
{
ReferenceResolver() = default;
/** Resolve references: for each reference, look for a matching
* anchor, and copy its contents to the ref node.
*
* @p tree the subject tree
*
* @p clear_anchors whether to clear existing anchors after
* resolving
*
* This method first does a full traversal of the tree to gather
* all anchors and references in a separate collection, then it
* goes through that collection to locate the names, which it does
* by obeying the YAML standard diktat that "an alias node refers
* to the most recent node in the serialization having the
* specified anchor"
*
* So, depending on the number of anchor/alias nodes, this is a
* potentially expensive operation, with a best-case linear
* complexity (from the initial traversal). This potential cost is
* one of the reasons for requiring an explicit call.
*
* The @ref Tree has an `Tree::resolve()` overload set forwarding
* here. Previously this operation was done there, using a
* discarded object; using this separate class offers opportunity
* for reuse of the object.
*
* @warning resolving references opens an attack vector when the
* data is malicious or severely malformed, as the tree can expand
* exponentially. See for example the [Billion Laughs
* Attack](https://en.wikipedia.org/wiki/Billion_laughs_attack).
*
*/
void resolve(Tree *tree, bool clear_anchors=true);
public:
/** @cond dev */
struct RefData
{
NodeType type;
id_type node;
id_type prev_anchor;
id_type target;
id_type parent_ref;
id_type parent_ref_sibling;
};
void reset_(Tree *t_);
void resolve_();
void gather_anchors_and_refs_();
void gather_anchors_and_refs__(id_type n);
id_type count_anchors_and_refs_(id_type n);
id_type lookup_(RefData const* C4_RESTRICT ra);
Tree *C4_RESTRICT m_tree;
/** We're using this stack purely as an array. */
detail::stack<RefData> m_refs;
/** @endcond */
};
/** @} */
} // namespace ryml
} // namespace c4
#endif // _C4_YML_REFERENCE_RESOLVER_HPP_

View File

@@ -25,16 +25,17 @@ void write(c4::yml::NodeRef *n, std::map<K, V, Less, Alloc> const& m)
}
}
/** read the node members, assigning into the existing map. If a key
* is already present in the map, then its value will be
* move-assigned. */
template<class K, class V, class Less, class Alloc>
bool read(c4::yml::ConstNodeRef const& n, std::map<K, V, Less, Alloc> * m)
{
K k{};
V v{};
for(auto const& C4_RESTRICT ch : n)
{
K k{};
ch >> c4::yml::key(k);
ch >> v;
m->emplace(std::make_pair(std::move(k), std::move(v)));
ch >> (*m)[k];
}
return true;
}

View File

@@ -17,31 +17,37 @@ template<class V, class Alloc>
void write(c4::yml::NodeRef *n, std::vector<V, Alloc> const& vec)
{
*n |= c4::yml::SEQ;
for(auto const& v : vec)
for(V const& v : vec)
n->append_child() << v;
}
/** read the node members, overwriting existing vector entries. */
template<class V, class Alloc>
bool read(c4::yml::ConstNodeRef const& n, std::vector<V, Alloc> *vec)
{
vec->resize(n.num_children());
C4_SUPPRESS_WARNING_GCC_WITH_PUSH("-Wuseless-cast")
vec->resize(static_cast<size_t>(n.num_children()));
C4_SUPPRESS_WARNING_GCC_POP
size_t pos = 0;
for(auto const ch : n)
ch >> (*vec)[pos++];
for(ConstNodeRef const child : n)
child >> (*vec)[pos++];
return true;
}
/** specialization: std::vector<bool> uses std::vector<bool>::reference as
/** read the node members, overwriting existing vector entries.
* specialization: std::vector<bool> uses std::vector<bool>::reference as
* the return value of its operator[]. */
template<class Alloc>
bool read(c4::yml::ConstNodeRef const& n, std::vector<bool, Alloc> *vec)
{
vec->resize(n.num_children());
C4_SUPPRESS_WARNING_GCC_WITH_PUSH("-Wuseless-cast")
vec->resize(static_cast<size_t>(n.num_children()));
C4_SUPPRESS_WARNING_GCC_POP
size_t pos = 0;
bool tmp = false;
for(auto const ch : n)
bool tmp = {};
for(ConstNodeRef const child : n)
{
ch >> tmp;
child >> tmp;
(*vec)[pos++] = tmp;
}
return true;

View File

@@ -0,0 +1,84 @@
#ifndef _C4_YML_TAG_HPP_
#define _C4_YML_TAG_HPP_
#include <c4/yml/common.hpp>
namespace c4 {
namespace yml {
class Tree;
/** @addtogroup doc_tag_utils
*
* @{
*/
#ifndef RYML_MAX_TAG_DIRECTIVES
/** the maximum number of tag directives in a Tree */
#define RYML_MAX_TAG_DIRECTIVES 4
#endif
/** the integral type necessary to cover all the bits marking node tags */
using tag_bits = uint16_t;
/** a bit mask for marking tags for types */
typedef enum : tag_bits {
TAG_NONE = 0,
// container types
TAG_MAP = 1, /**< !!map Unordered set of key: value pairs without duplicates. @see https://yaml.org/type/map.html */
TAG_OMAP = 2, /**< !!omap Ordered sequence of key: value pairs without duplicates. @see https://yaml.org/type/omap.html */
TAG_PAIRS = 3, /**< !!pairs Ordered sequence of key: value pairs allowing duplicates. @see https://yaml.org/type/pairs.html */
TAG_SET = 4, /**< !!set Unordered set of non-equal values. @see https://yaml.org/type/set.html */
TAG_SEQ = 5, /**< !!seq Sequence of arbitrary values. @see https://yaml.org/type/seq.html */
// scalar types
TAG_BINARY = 6, /**< !!binary A sequence of zero or more octets (8 bit values). @see https://yaml.org/type/binary.html */
TAG_BOOL = 7, /**< !!bool Mathematical Booleans. @see https://yaml.org/type/bool.html */
TAG_FLOAT = 8, /**< !!float Floating-point approximation to real numbers. https://yaml.org/type/float.html */
TAG_INT = 9, /**< !!float Mathematical integers. https://yaml.org/type/int.html */
TAG_MERGE = 10, /**< !!merge Specify one or more mapping to be merged with the current one. https://yaml.org/type/merge.html */
TAG_NULL = 11, /**< !!null Devoid of value. https://yaml.org/type/null.html */
TAG_STR = 12, /**< !!str A sequence of zero or more Unicode characters. https://yaml.org/type/str.html */
TAG_TIMESTAMP = 13, /**< !!timestamp A point in time https://yaml.org/type/timestamp.html */
TAG_VALUE = 14, /**< !!value Specify the default value of a mapping https://yaml.org/type/value.html */
TAG_YAML = 15, /**< !!yaml Specify the default value of a mapping https://yaml.org/type/yaml.html */
} YamlTag_e;
RYML_EXPORT YamlTag_e to_tag(csubstr tag);
RYML_EXPORT csubstr from_tag(YamlTag_e tag);
RYML_EXPORT csubstr from_tag_long(YamlTag_e tag);
RYML_EXPORT csubstr normalize_tag(csubstr tag);
RYML_EXPORT csubstr normalize_tag_long(csubstr tag);
RYML_EXPORT csubstr normalize_tag_long(csubstr tag, substr output);
RYML_EXPORT bool is_custom_tag(csubstr tag);
struct RYML_EXPORT TagDirective
{
/** Eg <pre>!e!</pre> in <pre>%TAG !e! tag:example.com,2000:app/</pre> */
csubstr handle;
/** Eg <pre>tag:example.com,2000:app/</pre> in <pre>%TAG !e! tag:example.com,2000:app/</pre> */
csubstr prefix;
/** The next node to which this tag directive applies */
id_type next_node_id;
bool create_from_str(csubstr directive_); ///< leaves next_node_id unfilled
bool create_from_str(csubstr directive_, Tree *tree);
size_t transform(csubstr tag, substr output, Callbacks const& callbacks) const;
};
struct RYML_EXPORT TagDirectiveRange
{
TagDirective const* C4_RESTRICT b;
TagDirective const* C4_RESTRICT e;
C4_ALWAYS_INLINE TagDirective const* begin() const noexcept { return b; }
C4_ALWAYS_INLINE TagDirective const* end() const noexcept { return e; }
};
/** @} */
} // namespace yml
} // namespace c4
#endif /* _C4_YML_TAG_HPP_ */

File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More