Support CodeMirror syntax highlighting on RTL wikis
Open, MediumPublic8 Estimated Story Points

Description

From Eranroz, on Meta talk page:

For right to left languages it is broken, for example the text direction is always left to right and browser switch text direction functionality ([Ctrl+Shift/Ctrl+Shift+X]) doesn't change the text direction compared to simple textarea. This is very important to RTL users as it is quite common to handle LTR embedded text within RTL content.

Relevant links:

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Some RTL bugs may have been fixed by https://gerrit.wikimedia.org/r/#/c/380678/. Let's re-test and see if there are still problems.

Some RTL bugs may have been fixed by https://gerrit.wikimedia.org/r/#/c/380678/. Let's re-test and see if there are still problems.

Well, I tried to check this on testwiki, using javascript, but still on right aligned test, if pressing keyboard End key, the cursor remains at the start of the line.

Some RTL bugs may have been fixed by https://gerrit.wikimedia.org/r/#/c/380678/. Let's re-test and see if there are still problems.

Well, I tried to check this on testwiki, using javascript, but still on right aligned test, if pressing keyboard End key, the cursor remains at the start of the line.

You didn't have to do that. Can you instead test on https://he.wikipedia.beta.wmflabs.org/wiki/יוסף_הלוי (make sure your JavaScript doesn't interfere) and tell us what are the major problems with it? It seems a little better than before but I think I still see some problems.

The goal is to figure out how disruptive the bugs are. Is it something that will stop people from editing completely? Is it only a bug that occurs sometimes? Is it something that only occurs when editing templates? Etc.

You didn't have to do that. Can you instead test on https://he.wikipedia.beta.wmflabs.org/wiki/יוסף_הלוי (make sure your JavaScript doesn't interfere) and tell us what are the major problems with it? It seems a little better than before but I think I still see some problems.

Thank you for this link. I tried, and it's even worse than with javascript today. The problems are:

  1. Ctrl Shift and Alt Shift do not work,
  2. The cursor position does not fit the real editing position.
  3. You can't put the cursor to any place with a mouse.
  4. The line beginning is still on the opposite side.
  5. Pressing the Home or End keyboard keys puts the cursor to different line of the document.
  6. Etc.

The good thing is that now, in opposite to a month before, you can work on rtl alignment, with rtl text only, and see the colours.
Sorry, but it still can't be used. Thank you again.

TBolliger renamed this task from Syntax highlighting - RTL problems to Support CodeMirror syntax highlighting on RTL wikis.Mar 2 2018, 12:49 AM

Marking as Stalled due to CodeMirror library work required by external developers.

@Esanders Do you think it's possible for us to take away cursor control from CM and let the browser handle it for the classic/advanced editors in the way that VE does it? CM doesn't provide a config option for that in the library and insists on handling cursor movements itself, which it is not very good at.

That would be feasible, although with T190720 you may lose IE11 support.

Another issue might be scroll sync, as on some devices there is a hardware accelerated scrolling animation that may not fire enough events to keep the two editors in sync during the animation: http://jsfiddle.net/5kth4fnx/47/

@Esanders Do you think it's possible for us to take away cursor control from CM and let the browser handle it for the classic/advanced editors in the way that VE does it? CM doesn't provide a config option for that in the library and insists on handling cursor movements itself, which it is not very good at.

That would be feasible, although with T190720 you may lose IE11 support.

Making the feature available to some is better than making it available to none. Also with T191093, it looks like we might have to drop support for IE and Edge altogether.

Another issue might be scroll sync, as on some devices there is a hardware accelerated scrolling animation that may not fire enough events to keep the two editors in sync during the animation: http://jsfiddle.net/5kth4fnx/47/

That does not sound pleasant. How does VE take care of that?

This demo, which uses CodeMirror 5.60, seems to work fine with RTL. Editing on HE Wikipedia, which runs 5.58.3, doesn't seem to work well. Is the issue actually with CodeMirror?

Change 965009 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/CodeMirror@master] Enable support for RTL when using CodeMirror 6

https://gerrit.wikimedia.org/r/965009

TheresNoTime changed the task status from Stalled to Open.Oct 23 2023, 12:34 PM
TheresNoTime subscribed.

Unstalling, work in progress

Change 965009 merged by jenkins-bot:

[mediawiki/extensions/CodeMirror@master] Enable support for RTL when using CodeMirror 6

https://gerrit.wikimedia.org/r/965009

If anyone wants to test this, CodeMirror 6 is installed on https://en-rtl.wikipedia.beta.wmflabs.org and https://he.wikipedia.beta.wmflabs.org. We have not done any additional work yet for RTL support other than what comes stock with CodeMirror, but two things I'd like to look into are:

  • Add bidi isolation to tags such as <div class="foobar">
  • Use LTR inside the content of applicable tags like <div dir="ltr">. We probably can't do the same if the direction is changed with a template, unfortunately.

Are there other things I should be looking at? Most of the things mentioned at T170001#3676449 by @IKhitron should be fixed now.

Any help with testing is appreciated :) Thank you!

If anyone wants to test this, CodeMirror 6 is installed on https://en-rtl.wikipedia.beta.wmflabs.org and https://he.wikipedia.beta.wmflabs.org. We have not done any additional work yet for RTL support other than what comes stock with CodeMirror, but two things I'd like to look into are:

  • Add bidi isolation to tags such as <div class="foobar">
  • Use LTR inside the content of applicable tags like <div dir="ltr">. We probably can't do the same if the direction is changed with a template, unfortunately.

Are there other things I should be looking at? Most of the things mentioned at T170001#3676449 by @IKhitron should be fixed now.

Any help with testing is appreciated :) Thank you!

Hello and thanks. I think maybe we should suggest to hewiki to reopen the CodeMirror for a trial, as a part of being this wiki the usual "rtl testing field", and if the community agrees, get much more comments and possibly some bugs.

Hello and thanks. I think maybe we should suggest to hewiki to reopen the CodeMirror for a trial, as a part of being this wiki the usual "rtl testing field", and if the community agrees, get much more comments and possibly some bugs.

That sounds great! Given that CodeMirror is an opt-in feature, do you think we need to propose using hewiki as a pilot wiki, or can we just ship it and ask for feedback? Of course, if the feedback is "remove this!", we will do so :) But we are committed to iterating on the feature until it satisfies the needs of RTL users.

Note you can test it now on Beta, i.e. https://he.wikipedia.beta.wmflabs.org/wiki/%D7%99%D7%A4%D7%94_%D7%AA%D7%95%D7%A1%D7%99%D7%94_%D7%9B%D7%94%D7%9F?action=edit (make sure to enable it using the Highlighter icon in the toolbar)

"Opt-in" means that newcomers to enwiki, for example, do not get it by default, @MusikAnimal?

"Opt-in" means that newcomers to enwiki, for example, do not get it by default, @MusikAnimal?

Correct. I'm still work on writing the user docs, but in this case, only WikiEditor is effected. CodeMirror would only be enabled if you turn it on using the highlighter icon in the WikiEditor toolbar.

In this case we definitely should consult to community first, because otherwise, from my experience, nobody will pay attention. A week discussion will put it on the radar.

In this case we definitely should consult to community first, because otherwise, from my experience, nobody will pay attention. A week discussion will put it on the radar.

Great. Thanks @IKhitron. That said @MusikAnimal let's collaborate on the announcement concerning the deployment.

Change #1019432 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/CodeMirror@master] Add $wgCodeMirrorRTL to control rollout to RTL wikis

https://gerrit.wikimedia.org/r/1019432

Change #1019432 merged by jenkins-bot:

[mediawiki/extensions/CodeMirror@master] Add $wgCodeMirrorRTL to control rollout to RTL wikis

https://gerrit.wikimedia.org/r/1019432

Change #1037658 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[operations/mediawiki-config@master] [beta] Enable CodeMirrorRTL on all wikis

https://gerrit.wikimedia.org/r/1037658

I would like to work on the Mod-Shift-x key binding to toggle between LTR and RTL. Before doing that, have we heard any feedback from hewiki where CodeMirror 6 has already been deployed?

I would like to work on the Mod-Shift-x key binding to toggle between LTR and RTL. Before doing that, have we heard any feedback from hewiki where CodeMirror 6 has already been deployed?

That would be great! I had a chat with some RTL-savvy engineers at the Hackathon and they mentioned CodeMirror was missing a toggle to force the direction of the text (as in the entire textarea, which I think is what you meant too). The 2017 editor has this option, so CodeMirror should as well. Just a keystroke is fine for now, but I've also added this as a to-do for the preferences panel (T359498).

From Hebrew Wikipedia, I only heard from people who accidentally were using CodeMirror 5 in the 2017 editor (a bug that has since been fixed). No one else said a thing, which usually means no glaring issues :) But given so many expected it to work in the 2017 editor, I am holding back deployment until T357482 is complete -- which is nearly done and I hope to revisit next week.

Change #1040441 had a related patch set uploaded (by Bhsd; author: Bhsd):

[mediawiki/extensions/CodeMirror@master] codemirror.mediawiki.js: add `Mod-Shift-x`

https://gerrit.wikimedia.org/r/1040441

I didn't see any complains this time on hewiki forums.

@MusikAnimal @IKhitron Thanks for your replies! I have submitted the patch for the key binding, and the part about Compartment may also be useful to T359498.