# Javascript Tokeniser

**URL:** <https://forum.juce.com/t/javascript-tokeniser/25163>\
**Category:** General JUCE discussion\
**Created:** [November 17, 2017, 8:53am UTC](https://forum.juce.com/t/javascript-tokeniser/25163 "2017-11-17T08:53:43Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![rebbur](https://avatars.discourse-cdn.com/v4/letter/r/ecae2f/32.png) [@rebbur](https://forum.juce.com/u/rebbur)\
**Post date:** [November 17, 2017, 8:53am UTC](https://forum.juce.com/t/javascript-tokeniser/25163/1 "2017-11-17T08:53:43Z")

</div>

I’m looking for a Javascript Tokeniser, it seems to be missing in JUCE. We have a JavascriptEngine but no tokeniser for it. Oddly enough i found a LUA tokeniser in the JUCE repository but LUA isn’t actually supported as scripting language right?

I could use the CPlusPlusCodeTokeniser of course as a starting point but i actually expected to be part of the CodeBase. A  
Anyone made a Javascript tokeniser i could use?

---

<div class="post-metadata">

**Author:** ![jrlanglois](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/jrlanglois/32/21372_2.png) [@jrlanglois](https://forum.juce.com/u/jrlanglois)\
**Post date:** [November 17, 2017, 2:47pm UTC](https://forum.juce.com/t/javascript-tokeniser/25163/2 "2017-11-17T14:47:30Z")

</div>

Funnily enough I was looking for the same thing recently! Would be cool to have a JSON tokeniser as well. Both would be really fitting with what JUCE offers.

---

<div class="post-metadata">

**Author:** ![pckerneis](https://avatars.discourse-cdn.com/v4/letter/p/97f17d/32.png) [@pckerneis](https://forum.juce.com/u/pckerneis)\
**Post date:** [December 19, 2017, 4:22pm UTC](https://forum.juce.com/t/javascript-tokeniser/25163/3 "2017-12-19T16:22:42Z")

</div>

I’ve tweaked the Lua and C++ tokenizers and I’ve came to this. Please let me know if you’d change anything.

[https://github.com/pckerneis/JsTokeniser](https://github.com/pckerneis/JsTokeniser)

---

<div class="post-metadata">

**Author:** ![rebbur](https://avatars.discourse-cdn.com/v4/letter/r/ecae2f/32.png) [@rebbur](https://forum.juce.com/u/rebbur)\
**Post date:** [December 20, 2017, 2:40pm UTC](https://forum.juce.com/t/javascript-tokeniser/25163/4 "2017-12-20T14:40:24Z")

</div>

@pckerneis, i also took the C++ tokeniser as base and came up with more or less the same as you did. I did add the root methods of the JavaScriptEngine as well, so methods like. exec, eval, trace, charToInt, parseInt, typeof and parsefloat.

---

<div class="post-metadata">

**Author:** ![rory](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/rory/32/534_2.png) [@rory](https://forum.juce.com/u/rory)\
**Post date:** [February 15, 2019, 9:01pm UTC](https://forum.juce.com/t/javascript-tokeniser/25163/5 "2019-02-15T21:01:48Z")

</div>

Hey @pckerneis, do you mind if I use this? You don’t have any license on your repo…

---

<div class="post-metadata">

**Author:** ![pckerneis](https://avatars.discourse-cdn.com/v4/letter/p/97f17d/32.png) [@pckerneis](https://forum.juce.com/u/pckerneis)\
**Post date:** [February 16, 2019, 9:38am UTC](https://forum.juce.com/t/javascript-tokeniser/25163/6 "2019-02-16T09:38:35Z")

</div>

Yeah sorry, of course feel free to use or modify it.

---

<div class="post-metadata">

**Author:** ![rory](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/rory/32/534_2.png) [@rory](https://forum.juce.com/u/rory)\
**Post date:** [February 16, 2019, 6:56pm UTC](https://forum.juce.com/t/javascript-tokeniser/25163/7 "2019-02-16T18:56:16Z")

</div>

Your name will remain on the source code. Much appreicated 😉

---

<div class="post-metadata">

**Author:** ![asimilon](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/asimilon/32/4235_2.png) [@asimilon](https://forum.juce.com/u/asimilon)\
**Post date:** [February 17, 2019, 4:18pm UTC](https://forum.juce.com/t/javascript-tokeniser/25163/8 "2019-02-17T16:18:39Z")

</div>

> [@Rory](#):
>
> You don’t have any license on your repo

[It does now](https://github.com/pckerneis/JsTokeniser/blob/master/LICENSE) and it’s GPL3.0, which may or may not be a problem for you.

---

<div class="post-metadata">

**Author:** ![rory](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.juce.com/rory/32/534_2.png) [@rory](https://forum.juce.com/u/rory)\
**Post date:** [February 17, 2019, 4:35pm UTC](https://forum.juce.com/t/javascript-tokeniser/25163/9 "2019-02-17T16:35:40Z")

</div>

Not a problem.
