# How to compare UTF-8 strings?

**URL:** <https://forum.juce.com/t/how-to-compare-utf-8-strings/33501>\
**Category:** Development\
**Created:** [May 21, 2019, 9:22am UTC](https://forum.juce.com/t/how-to-compare-utf-8-strings/33501 "2019-05-21T09:22:28Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![MBO](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/mbo/32/5045_2.png) [@MBO](https://forum.juce.com/u/MBO)\
**Post date:** [May 21, 2019, 9:22am UTC](https://forum.juce.com/t/how-to-compare-utf-8-strings/33501/1 "2019-05-21T09:22:28Z")

</div>

I’m wondering how to deal with UTF-8 string comparison in JUCE in the most efficient way. Is the only solution to normalize strings using some external library like ICU or utf8proc and compare them after?

---

<div class="post-metadata">

**Author:** ![jrlanglois](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/jrlanglois/32/21372_2.png) [@jrlanglois](https://forum.juce.com/u/jrlanglois)\
**Post date:** [May 21, 2019, 9:42am UTC](https://forum.juce.com/t/how-to-compare-utf-8-strings/33501/2 "2019-05-21T09:42:33Z")

</div>

`juce::String` already does this work for you. By default, its internals work with UTF-8. In addition, this is a configurable option: the `String` class can internally store data as either UTF-16 and UTF-32 if you so need that (which is pretty rare!).

---

<div class="post-metadata">

**Author:** ![jrlanglois](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/jrlanglois/32/21372_2.png) [@jrlanglois](https://forum.juce.com/u/jrlanglois)\
**Post date:** [May 21, 2019, 9:46am UTC](https://forum.juce.com/t/how-to-compare-utf-8-strings/33501/3 "2019-05-21T09:46:53Z")

</div>

I should add that you can convert between UTF8/16/32 using the `toUTFn` functions.

eg: [juce::String::toUTF8()](https://docs.juce.com/master/classString.html#aabef96ce5e32ca15473b3694309f2647)

---

<div class="post-metadata">

**Author:** ![daniel](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/daniel/32/790_2.png) [@daniel](https://forum.juce.com/u/daniel)\
**Post date:** [May 21, 2019, 9:57am UTC](https://forum.juce.com/t/how-to-compare-utf-8-strings/33501/4 "2019-05-21T09:57:50Z")

</div>

But would be nice to have a String compare, that matches Müller and Mueller optionally…

---

<div class="post-metadata">

**Author:** ![jrlanglois](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/jrlanglois/32/21372_2.png) [@jrlanglois](https://forum.juce.com/u/jrlanglois)\
**Post date:** [May 21, 2019, 10:06am UTC](https://forum.juce.com/t/how-to-compare-utf-8-strings/33501/5 "2019-05-21T10:06:45Z")

</div>

Yeah, that would mean having language databases like what ICU employs, and break iterators to go along with it. It’s pretty complicated stuff!

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://forum.juce.com/u/jules)\
**Post date:** [May 21, 2019, 10:07am UTC](https://forum.juce.com/t/how-to-compare-utf-8-strings/33501/6 "2019-05-21T10:07:51Z")

</div>

On OSX there’s a String::convertToPrecomposedUnicode() method, but unfortunately there’s no practical way for us to embed cross-platform functionality to do that kind of operation. There are unicode libraries that do it, but they’re enormous and complicated to build.

---

<div class="post-metadata">

**Author:** ![jrlanglois](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/jrlanglois/32/21372_2.png) [@jrlanglois](https://forum.juce.com/u/jrlanglois)\
**Post date:** [May 21, 2019, 10:15am UTC](https://forum.juce.com/t/how-to-compare-utf-8-strings/33501/7 "2019-05-21T10:15:33Z")

</div>

Enormous and complicated also applies to use and maintain. 🙂

Take ICU for example; if you don’t want to the entire data library and all of ICU’s features, it’s very difficult to make that work correctly on all systems.

- You have to maintain data files containing subsets
- You have to make sure that the data works with specific versions of ICU (due to poor backward and forward compatibility… speaking from experience here!)
- You have to suffer the overhead of representing 2 separate data packages when dealing with LE and BE systems.

Lots can go wrong very easily… It’s unfortunate that the design and architecture for ICU is so poor.

---

<div class="post-metadata">

**Author:** ![daniel](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/daniel/32/790_2.png) [@daniel](https://forum.juce.com/u/daniel)\
**Post date:** [May 21, 2019, 10:19am UTC](https://forum.juce.com/t/how-to-compare-utf-8-strings/33501/8 "2019-05-21T10:19:18Z")

</div>

I absolutely agree, it is a can of worms, and not really in the focus of an audio framework.

I get horrified just thinking about, how many spellings I saw for Tschaikowski, Tchaikovsky or any other… So we Germans with 3 umlauts and one special ß are the minor problem 😉

---

<div class="post-metadata">

**Author:** ![MBO](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/mbo/32/5045_2.png) [@MBO](https://forum.juce.com/u/MBO)\
**Post date:** [May 21, 2019, 11:26am UTC](https://forum.juce.com/t/how-to-compare-utf-8-strings/33501/9 "2019-05-21T11:26:09Z")

</div>

Thanks for the answers!

I need to compare for example Müller and Myller, so right now using `String::compare` does not give the expected result… I understand these strings should be normalized first and then compared… For example in Qt there is `String::normalized` method for this purpose. I’d like to avoid using ICU for the reasons already mentioned.

Of course I am not suggesting here that JUCE should be extended with some heavyweight classes with all national subsets, I’d rather want to find a simple workaround, don’t need to compare ü and ue or ß and ss.

---

<div class="post-metadata">

**Author:** ![jrlanglois](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/jrlanglois/32/21372_2.png) [@jrlanglois](https://forum.juce.com/u/jrlanglois)\
**Post date:** [May 21, 2019, 12:56pm UTC](https://forum.juce.com/t/how-to-compare-utf-8-strings/33501/10 "2019-05-21T12:56:19Z")

</div>

In fairness, juce wraps the iOS and macOS APIs which does the job well enough (like Jules mentioned above - `String::convertToPrecomposedUnicode()`).

Its API _could_ be extended to wrap the remaining native functions in a semi-consistent way:

- Windows ([https://docs.microsoft.com/en-us/windows/desktop/api/winnls/nf-winnls-normalizestring](https://docs.microsoft.com/en-us/windows/desktop/api/winnls/nf-winnls-normalizestring))
- Android ([https://developer.android.com/reference/java/text/Normalizer](https://developer.android.com/reference/java/text/Normalizer))

I’m pretty sure in Linux’s case you would need to use ICU. Don’t quote me on that - research is coming up dry for POSIX APIs.

Definitely a can of worms that would involve many unit tests, imo.

---

<div class="post-metadata">

**Author:** ![MBO](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/mbo/32/5045_2.png) [@MBO](https://forum.juce.com/u/MBO)\
**Post date:** [May 21, 2019, 1:33pm UTC](https://forum.juce.com/t/how-to-compare-utf-8-strings/33501/11 "2019-05-21T13:33:15Z")

</div>

Thanks, that’s a good starting point.

Unfortunately I need to have it on Linux in the first place (embedded devices). I’ll try to check some less heavy alternatives to ICU first.

---

<div class="post-metadata">

**Author:** ![MBO](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/mbo/32/5045_2.png) [@MBO](https://forum.juce.com/u/MBO)\
**Post date:** [November 1, 2019, 5:19pm UTC](https://forum.juce.com/t/how-to-compare-utf-8-strings/33501/12 "2019-11-01T17:19:54Z")

</div>

If somebody is interested in comparing UTF-8 strings according to UTF-8 standards and successful uppercase/lowercase operations, then utf8proc library works as expected on all platforms and (at least for me) is a sufficient replacement for ICU library. It adds only 300KB to release builds.
