Skip to main content

On Sale: GamesAssetsToolsTabletopComics
Indie game storeFree gamesFun gamesHorror games
Game developmentAssetsComics
SalesBundles
Jobs
TagsGame Engines

Japanese punctuation cannot be displayed in the first letter of a sentence

A topic by ChuanJianHua created Dec 01, 2020 Views: 655 Replies: 24
Viewing posts 1 to 21

Time is urgent, hope to solve it soon, thank you!

Developer (1 edit)

Hello! I haven't fully implemented this yet (I just started learning Japanese myself, so I see the importance of this) and while better text rag is planned for STM someday, I won't be able to get that in urgently.

So! As a quick workaround, here's two steps to follow:

First, inside of SuperTextMesh.cs, there's a variable named "linebreakFriendlyChars" declared on line 1106 change that line to this:

private char[] linebreakFriendlyChars = new char[]{' ', '\n', '\t', '-', '\u00AD', '\u200A', '\u200B', '。', '〜', '、', ',', '…', '‥'};

This will allow STM to insert automatic linebreaks at Japanese punctuation, if it's available.


Unfortunately, this doesn't solve the above circumstance where no punctuation happens before the first line break, so the second thing to do is utilize zero-width spaces! They're easy to lose track of, so you can use the tag <u=200B> in STM to insert a zero-width space in a spot where a linebreak is safe to perform. You could also just manually insert a line break with <br> if you want a line break to happen for sure at a certain location.


In the future, I'd love to have STM be able to tell where Japanese words start and stop, but without spaces it's a bit of a challenge... I'd probably have to implement a dictionary function! I'm going to write this down, I think I just thought of a way to make this happen in the future... but for an urgent problem, the above solution will work! The idea I had was... if I implement a dictionary, I could have STM automatically put a zero-width space between every character... but in reality, it'd probably be better to have a script like this run on your entire game's script before it even gets to STM, so a dictionary doesn't need to be parsed every time STM needs to rebuild... Is it possible to run a method like that on your game's script?

Sorry that last paragraph is a bit rambling! I really should be asleep, but I hope this helps!

Developer

Hey!


I finally added this as a feature: https://twitter.com/KaiClavier/status/1441149108744454155


I'll be adding this to the next release of STM, which will hopefully be soon! But please let me know if you want a beta version. Sorry it took a year!

(1 edit)

congratulations!I found an error when using the previous method, an "ArgumentOutOfRangeException: startIndex cannot be larger than length of string. ", in AssemleLeftoverText function. Can you help me solve it or teach me the new method ?

Developer

Hello,

The new version of STM has been released! Is this error happening in v1.12.0? If not, please upgrade and let me know if the bug persists, and I'll do my best to fix it!

Yes, I just updated the version yesterday

Developer (1 edit)

Please send me a screenshot of the Unity Editor with the full inspector for Super Text Mesh open so I can reproduce the error! I want to see the text and autoWrap values specifically.

text is  : 

ホ<u=200B>ッ<u=200B>ツ<u=200B>先<u=200B>生<u=200B>は<u=200B>昔<u=200B>こ<u=200B>こ<u=200B>の<u=200B>名<u=200B>誉<u=200B>市<u=200B>民<u=200B>の<u=200B>称<u=200B>号<u=200B>を<u=200B>取<u=200B>っ<u=200B>た<u=200B>こ<u=200B>と<u=200B>も<u=200B>あ<u=200B>っ<u=200B>た<u=200B>ん<u=200B>だ<u=200B>っ<u=200B>た<u=200B>な…<u=200B>え<u=200B>ー<u=200B>と、<u=200B>あ<u=200B>れ<u=200B>は<u=200B>何<u=200B>年<u=200B>前<u=200B>の<u=200B>こ<u=200B>と<u=200B>だ<u=200B>っ<u=200B>た<u=200B>か<u=200B>な<u=200B>?<u=200B>覚<u=200B>え<u=200B>て<u=200B>は<u=200B>い<u=200B>る<u=200B>ん<u=200B>だ<u=200B>が、<u=200B>具<u=200B>体<u=200B>的<u=200B>な<u=200B>こ<u=200B>と<u=200B>が<u=200B>思<u=200B>い<u=200B>出<u=200B>せ<u=200B>な<u=200B>い…<u=200B>も<u=200B>う<u=200B>年<u=200B>だ<u=200B>な。

inspector  : 

Developer

Got it! I'll make sure to publish the fix soon, but for now, make these 3 changes to SuperTextMesh,cs:


Line 2947:

deletedChars += closingIndex-i;


Line 3002:

ParseText_info.rawIndex = i + deletedChars + 1;


Line 3861 - put this if statement in front of it:

if(cutoffPosition <= preParsedText.Length)

Thank you! It works very well .

Ran into a similar problem again,When I use Chinese, <u=200B> is truncated,the text is as follows :

希<u=200B>望<u=200B>你<u=200B>赏<u=200B>脸,<u=200B>等<u=200B><c=red><u=200B>玉<u=200B>米<u=200B>争<u=200B>霸<u=200B>赛<u=200B></c><u=200B>为<u=200B>我<u=200B>打<u=200B>开<u=200B>名<u=200B>气,<u=200B><c=red><u=200B>我<u=200B>就<u=200B>能<u=200B>在<u=200B>诺<u=200B>克<u=200B>提<u=200B>斯<u=200B>拥<u=200B>有<u=200B>自<u=200B>己<u=200B>的<u=200B>专<u=200B>卖<u=200B>店<u=200B>了<u=200B>!<u=200B></c><u=200B>这<u=200B>位<u=200B>美<u=200B>丽<u=200B>的<u=200B>小<u=200B>姐,<u=200B>有<u=200B>兴<u=200B>趣<u=200B>参<u=200B>加<u=200B>一<u=200B>下<u=200B>吗<u=200B>?

Inspector as above,Thanks for your help!

Developer

Aw darn I thought I caught this with the first two lines of code I changed in the last fix, but I'll see if I can still reproduce this soon!

Developer

Okay, I think I've got it. 


Try moving the "ParseText_info.rawIndex = i + deletedChars + 1;" line to happen before the "checkAgain" if statement that currently precedes it. (Around line 2981) It seems to be producing the correct results now.

Doesn't seem to work

Developer

Hmm, please send me an email so I can send you a test build, it's possible that there's something I changed without realizing. Sorry for the trouble.

I send the email, thank you.

I use the test build in an empty project, this is the result of my test, I guess I found the pattern.

Every time after a string of <c> characters, the raw Index will be misplaced by 1

Developer

Oh! I see now, the tag offset is still wrong... I should be able to fix this, I'll keep trying!

Developer

Please try changing this line:


ParseText_info.rawIndex = i + deletedChars + 1;


to this:


ParseText_info.rawIndex = i + deletedChars + allTags.Count +1;


This seems to behave correctly, I just hope I'm not mistaken again.

(1 edit)

Hi great product.  Using it on Critter Cove which is coming out on steam shortly.  Having this same issue with our Japanese translator and it appears to now work without issue with ! but not with periods as shown here.  So is there something I am doing wrong?  See the stranded Japanese . at the end of the sentence.

If its something we need to expand ourselves just point me in the right direction.  Basically our biggest issue right now is just grabbing the character previous to the punctuation and putting the line break before that so that we do not start sentences with punctuation as shown here.  Interesting that it seems you addressed this but for the life of me I cannot seem to consistently get it to work for some reason.  

Developer

Hi! Thanks for using STM, it means a ton!


I'm on vacation right now, but I'll give this a quick look...

Is the character '。' exactly? In STM, there's a long array of "linebreakUnfriendlyCharacters" which controls what characters can never appear at the start of a line, and this character is in it... Can you make sure there's no space/zero-width space before the '。'?


Also, is the variable "breakText" disabled in the STM inspector? Does disabling "insertHyphens" when using Japanese text make a diff at all?


Off the top of my head, those are two things I can think of to look at first! If neither of those work, I'll have to figure out what it could be... In that case, could you paste that exact string of text you're sending to STM here?

(6 edits)

* OK FOR NOW IGNORE THIS.  Appears the update did not take last time and so now I see the changes.  Sorry about that stupid unity said it was on the latest version but some of the files apparently did not take.  Fixing and will let you know if there are any issues.

WORKING AS DESIGNED NOW.  Again sorry.


Here is the exact text.  You can see there is no space at all at the end character '。'  Actual text is included at the bottom.

Toggling off and on the hyphen made no difference.  Thanks for the help going to take a look as well inside the code.

Also using a true type font so not sure if that has something to do with it???  Though it does appear to be working sometimes with ! inside of text ie not at the end.

Oh another oddity "linebreakUnfriendlyCharacters" could not find this while searching the code.   But I can find linebreakFriendlyChars so that seems odd.  Going to check if the update did not take at some point or if my custom changes interfered with that.

<c=green><a=center><i>ウキウキモード 起動</i></a></c><d=20>

全可動部 機能ヲ確認<d=20> 有機生命体ニ感謝シマス。

Developer

It's ok! It's strange, I've actually gotten a few errors where the solution was an update improperly applying... I've even run into it myself when re-importing despite seeing new files and making sure they were checked (in fact, I'm pretty sure that thanks to this, some code in another asset of mine hasn't gotten published so I'm planning a complete manual code comparison...) so... I'm not too sure why Unity's doing that, maybe something could be wrong with my metadata...?


Either way, glad it works!

It works ok, thank you.