c++ - C++20 with u8, char8_t and std::string

Question

C++11 brought us the u8 prefix for UTF-8 literals and I thought that was pretty cool a few years ago and peppered my code with things like this:

std::string myString = u8"●";

This is all fine and good, but the issue comes up in C++20 it doesn't seem to compile anymore because u8 creates a char8_t* and this is incompatible with std::string which just uses char.

Should I be creating a new utf8string? What's the consistent and correct way to do this kind of thing in a C++20 world where we have more explicit types that don't really match with the standard std::string?

question from:https://stackoverflow.com/questions/56833000/c20-with-u8-char8-t-and-stdstring

Answer 1

In addition to @lubgr's answer, the paper char8_t backward compatibility remediation (P1423) discusses several ways how to make std::string with char8_t character arrays.

Basically the idea is that you can cast the u8 char array into a "normal" char array to get the same behaviour as C++17 and before, you just have to be a bit more explicit. The paper discusses various ways to do this.

The most simple (but not fully zero overhead, unless you add more overloads) method that fits your usecase is probably the last one, i.e. introduce explicit conversion functions:

std::string from_u8string(const std::string &s) {
  return s;
}
std::string from_u8string(std::string &&s) {
  return std::move(s);
}
#if defined(__cpp_lib_char8_t)
std::string from_u8string(const std::u8string &s) {
  return std::string(s.begin(), s.end());
}
#endif