Re-Enable fmt module tests, update module-test.cc by MathewBensonCode · Pull Request #4702 · fmtlib/fmt

MathewBensonCode · 2026-03-05T11:44:34Z

Refactor test/CMakeLists.txt to enable testing for modules
The tests in module-test.cc seem to not have been updated in some time despite changes in the main library.
Wide String versions of several tests appear to be deprecated so have been commented out.
Refactored tests related to fmt::format_args that now requires lvalue references as opposed to direct values.
Tests for items that have since been removed from the library like fmt::is_char and fmt::localtime as far as I can tell
testing for the fmt::detail namespace. The MSVC github workflow could not differentiate the between the fmt::detail and fmt::v12::detail. Removed windows testing for this

Should close #4698

MathewBensonCode · 2026-03-06T13:05:56Z

@vitaut, what is the scope of testing that we require for testing the module target?

Do we just want to test the public API or the full API?

The tests for the regular library involve tests for the entire library as they are able to reach into the entire code base.

With the module based library, it brings up the issue of visible and invisible APIs.

In this pull request, Ideally I have just remove the "inner" code(and some outdated code), which is being tested by the regular library anyway, and just test that the expected public functionality works.

I have commented out the bits that would be deleted if we we're to take this approach.

The other approach would be to work out how to work with modules and macro based testing frameworks.

I'm doing some experiments with this, but it may need another pull request as the changes are likely to be drastic, like moving tests into headers, or making the tests part of the module, etc.

vitaut · 2026-03-07T00:42:49Z

We definitely don't need to test everything. A simple sanity test that exercises a small subset of the API should be sufficient. I would recommend splitting the changes into smaller PRs, e.g. focusing on one compiler (e.g. clang) first.

vitaut · 2026-03-07T00:46:37Z

That said, we shouldn't delete code from the module test if it is broken with modules. On the contrary - we should keep it as a regression test for the fix. It is OK to conditionally compile it until the fix is available.

MathewBensonCode · 2026-03-09T11:10:34Z

After further investigation, I have identified that the wide string versions of most tests in module-test.cc were not compiling due to the functionality being deprecated in the main library.

I have just commented them out with explanations in the comments, however, these may be candidates for removal as I observed that they we're not present in the other *-test.cc files.

Compiler Specific testing does not appear to be an issue in this case. The module-test.cc file was well written to only deal with module tests so likely just needed to be updated.

module-test run visible here:

https://github.com/fmtlib/fmt/actions/runs/22850169158/job/66276473337?pr=4702#step:7:1404

vitaut

Please apply cmake-format and revert unrelated changes.

vitaut · 2026-03-18T21:15:35Z

test/CMakeLists.txt

+    set(libs test-main fmt-module)
  else ()
-    set(libs test-main fmt)
+    set(libs test-main)


Why is fmt removed here?

fmt is removed here because it is already included in the test-main target

vitaut · 2026-03-18T21:16:59Z

test/CMakeLists.txt

+  if (MSVC)
+    target_compile_options(module-test PRIVATE /utf-8 /Zc:__cplusplus
+                                               /Zc:externConstexpr /Zc:inline)
+  endif ()


/utf-8 should be inherited from the fmt target and shouldn't be added here. What are the other options for?

vitaut · 2026-03-18T21:19:36Z

test/module-test.cc


-namespace detail {
-bool oops_detail_namespace_is_visible;
+#ifndef _WIN32 //windows msvc has issue with nested namespace


Let's disable modules on MSVC for now instead of adding these hacks.

I had taken inspiration from here

fmt/test/module-test.cc

Lines 509 to 520 in 4968433

TEST(module_test, compile_format_string) {

using namespace fmt::literals;

#ifdef __clang__

fmt::println("\033[0;33m[=disabled=] {}\033[0;0m",

"Clang 16.0 fails to import user-defined literals");

#else

EXPECT_EQ("42", fmt::format("{0:x}"_cf, 0x42));

EXPECT_EQ(L"42", fmt::format(L"{:}"_cf, 42));

EXPECT_EQ("4.2", fmt::format("{arg:3.1f}"_cf, "arg"_a = 4.2));

EXPECT_EQ(L" 42", fmt::format(L"{arg:>3}"_cf, L"arg"_a = L"42"));

#endif

}

I think it is better to isolate the one area with an issue and have the rest of the tests active.

Could there perhaps be a more acceptable way of doing this? I chose to use comments, but we could also use the print message stating that the test is disabled

vitaut · 2026-03-18T21:20:34Z

test/module-test.cc

+  int n = 42;
+  fmt::basic_format_args<fmt::format_context> args = fmt::make_format_args(n);


Why is this changed?

When using fmt::make_format_args(42) as it was before, there was an error stating that a suitable function couldn't be found.

Upon investigation I found that the tests we're written to use an lvalue, for example

fmt/test/xchar-test.cc

Lines 332 to 333 in 4968433

auto n = 1234567;

EXPECT_EQ("1~234~567", fmt::vformat(loc, "{:L}", fmt::make_format_args(n)));

.

So I used the same pattern everywhere that the fmt::make_format_args() was used in the module-test.cc and the issue went away.

It appears that there was a change and since the module-test.cc wasn't being actively used it fell behind

- Refactor test/CMakeLists.txt to enable testing for modules - The tests in `module-test.cc` seem to not have been updated in some time despite changes in the main library. - Wide String versions of several tests appear to be deprecated so have been commented out. - Refactored tests related to `fmt::format_args` that now requires lvalue references as opposed to direct values. These include: - testing for the fmt::detail namespace. The MSVC github workflow could not differentiate the between the fmt::detail and fmt::v12::detail and stderr. an exception was being thrown - Several tests for items that have since been removed from the library like `fmt::is_char` and `fmt::localtime` as far as I can tell

MathewBensonCode force-pushed the testmodules branch from 12b77b8 to 789a28f Compare March 5, 2026 18:42

MathewBensonCode marked this pull request as ready for review March 5, 2026 20:32

MathewBensonCode marked this pull request as draft March 6, 2026 04:32

vitaut mentioned this pull request Mar 7, 2026

Enable module test #4698

Open

MathewBensonCode force-pushed the testmodules branch from 789a28f to 8241bd6 Compare March 9, 2026 10:56

MathewBensonCode changed the title ~~Re-Enable fmt module tests, quarantine failing code~~ Re-Enable fmt module tests, update module-test.cc Mar 9, 2026

MathewBensonCode marked this pull request as ready for review March 9, 2026 11:02

MathewBensonCode mentioned this pull request Mar 9, 2026

Add C++-Modules Based Linux CI workflow #4708

Open

MathewBensonCode force-pushed the testmodules branch from 8241bd6 to b11dc45 Compare March 10, 2026 09:43

vitaut requested changes Mar 14, 2026

View reviewed changes

MathewBensonCode force-pushed the testmodules branch 2 times, most recently from 6f6bce7 to 9acd195 Compare March 16, 2026 13:19

vitaut requested changes Mar 18, 2026

View reviewed changes

MathewBensonCode force-pushed the testmodules branch from 9acd195 to acc41f6 Compare March 19, 2026 09:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Re-Enable fmt module tests, update module-test.cc#4702

Re-Enable fmt module tests, update module-test.cc#4702
MathewBensonCode wants to merge 1 commit intofmtlib:masterfrom
MathewBensonCode:testmodules

MathewBensonCode commented Mar 5, 2026 •

edited

Loading

Uh oh!

MathewBensonCode commented Mar 6, 2026

Uh oh!

vitaut commented Mar 7, 2026 •

edited

Loading

Uh oh!

vitaut commented Mar 7, 2026

Uh oh!

MathewBensonCode commented Mar 9, 2026 •

edited

Loading

Uh oh!

vitaut left a comment

Uh oh!

vitaut Mar 18, 2026

Uh oh!

MathewBensonCode Mar 19, 2026

Uh oh!

vitaut Mar 18, 2026

Uh oh!

vitaut Mar 18, 2026

Uh oh!

MathewBensonCode Mar 19, 2026

Uh oh!

vitaut Mar 18, 2026

Uh oh!

MathewBensonCode Mar 19, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	TEST(module_test, compile_format_string) {
	using namespace fmt::literals;
	#ifdef __clang__
	fmt::println("\033[0;33m[=disabled=] {}\033[0;0m",
	"Clang 16.0 fails to import user-defined literals");
	#else
	EXPECT_EQ("42", fmt::format("{0:x}"_cf, 0x42));
	EXPECT_EQ(L"42", fmt::format(L"{:}"_cf, 42));
	EXPECT_EQ("4.2", fmt::format("{arg:3.1f}"_cf, "arg"_a = 4.2));
	EXPECT_EQ(L" 42", fmt::format(L"{arg:>3}"_cf, L"arg"_a = L"42"));
	#endif
	}

		int n = 42;
		fmt::basic_format_args<fmt::format_context> args = fmt::make_format_args(n);

	auto n = 1234567;
	EXPECT_EQ("1~234~567", fmt::vformat(loc, "{:L}", fmt::make_format_args(n)));

Uh oh!

Conversation

MathewBensonCode commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MathewBensonCode commented Mar 6, 2026

Uh oh!

vitaut commented Mar 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vitaut commented Mar 7, 2026

Uh oh!

MathewBensonCode commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vitaut left a comment

Choose a reason for hiding this comment

Uh oh!

vitaut Mar 18, 2026

Choose a reason for hiding this comment

Uh oh!

MathewBensonCode Mar 19, 2026

Choose a reason for hiding this comment

Uh oh!

vitaut Mar 18, 2026

Choose a reason for hiding this comment

Uh oh!

vitaut Mar 18, 2026

Choose a reason for hiding this comment

Uh oh!

MathewBensonCode Mar 19, 2026

Choose a reason for hiding this comment

Uh oh!

vitaut Mar 18, 2026

Choose a reason for hiding this comment

Uh oh!

MathewBensonCode Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

MathewBensonCode commented Mar 5, 2026 •

edited

Loading

vitaut commented Mar 7, 2026 •

edited

Loading

MathewBensonCode commented Mar 9, 2026 •

edited

Loading

MathewBensonCode Mar 19, 2026 •

edited

Loading