Item7937 < Bugs

AppliesTo:	Component:	Priority:	CurrentState:	WaitingFor:	TargetRelease	ReleasedIn
Engine		Normal	Being Worked On		patch	6.1.1

-- TWiki:Main/CalvinSo

- 2021-12-23

Regex definition of anchor is not correct in TWiki.pm

1. Does not support anchor name with special character like '-' E.g. Testpage#this-is-an-anchor

2. Does not support anchor name in non-alphabet charcaters like '%2A' E.g. Testpage#%2A%32

3. Does not support Scroll To Text Fragment anchor like Testpage#:~:text=anchortext

Line 468

$regex{anchorRegex} = qr/\#[$regex{mixedAlphaNum}_]+/o;

Line 440

$regex{mixedAlphaNum} = $regex{mixedAlpha}.$regex{numeric};

E.g.

AnchorText 日本

These link works if it is an externalLink

https://develop.twiki.org/do/view/Bugs/Item7937#:~:text=AnchorText

https://develop.twiki.org/do/view/Bugs/Item7937#%E6%97%A5%E6%9C%AC

https://develop.twiki.org/do/view/Bugs/Item7937#this-is-an-anchor

These link doesn't work the whole URL is treated as new topic name

Item7937#:~:text=AnchorText

Item7937#%E6%97%A5%E6%9C%AC

Item7937#this-is-an-anchor

#Item7937

$regex{anchorRegex} = qr/\#(?:[$regex{mixedAlphaNum}_\-\+\.\'=~:!\$&*,;]|%[0-9a-fA-F]{2})+/o;

-- TWiki:Main.CalvinSo - 2021-12-23

This change crashes the WebChanges here, and prevents creating new bug topics:

Assertion failed! at /var/www/develop.twiki.org/twiki/twikisvn/core/lib/Assert.pm line 61. Assert::ASSERT('') called at /var/www/develop.twiki.org/twiki/twikisvn/core/lib/TWiki/Render.pm line 781 TWiki::Render::_handleWikiWord('TWiki::Render=HASH(0x2ef7f90)', 'Bugs', undef, 'VERBATIM', '"') called at /var/www/develop.twiki.org/twiki/twikisvn/core/lib/TWiki/Render.pm line 1956 TWiki::Render::getRenderedVersion('TWiki::Render=HASH(0x2ef7f90)', '\x{a}\x{a}<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//...', 'Bugs', 'ItemAUTOINC1') called at /var/www/develop.twiki.org/twiki/twikisvn/core/lib/TWiki/UI/Edit.pm line 363 TWiki::UI::Edit::init_edit('TWiki=HASH(0x1ae4118)', 'edit') called at /var/www/develop.twiki.org/twiki/twikisvn/core/lib/TWiki/UI/Edit.pm line 68 TWiki::UI::Edit::edit('TWiki=HASH(0x1ae4118)') called at /var/www/develop.twiki.org/twiki/twikisvn/core/lib/TWiki/UI.pm line 204 TWiki::UI::__ANON__() called at /usr/share/perl5/vendor_perl/Error.pm line 419 eval {...} called at /usr/share/perl5/vendor_perl/Error.pm line 411 Error::subs::try('CODE(0x1ac8c30)', 'HASH(0x314a888)') called at /var/www/develop.twiki.org/twiki/twikisvn/core/lib/TWiki/UI.pm line 279 TWiki::UI::execute('TWiki::Request=HASH(0x240a8f0)', 'CODE(0x1b051f8)', 'edit', 1) called at /var/www/develop.twiki.org/twiki/twikisvn/core/lib/TWiki/UI.pm line 180 TWiki::UI::handleRequest('TWiki::Request=HASH(0x240a8f0)') called at /var/www/develop.twiki.org/twiki/twikisvn/core/lib/TWiki/Engine/CGI.pm line 48 TWiki::Engine::CGI::run('TWiki::Engine::CGI=HASH(0x20cd228)') called at edit line 47.

This is the assertion for anchors containing #. The assertion fails with create new bug topic with topic VERBATIM and anchor ".

Offending line in lib/TWiki/Render.pm seems to be:

31058 CalvinSo $text =~ s/$STARTWW(?:($TWiki::regex{webNameRegex})\.)?($TWiki::regex{wikiWordRegex}|$TWiki::regex{abbrevRegex})($TWiki::regex{anchorRegex}?[^\s*.,!?;:)<|]+)?/_handleWikiWord( $this,$theWeb,$1,$2,$3)/geom;

The question mark after $TWiki::regex{anchorRegex} makes it optional, and you append a negative character class after that, e.g. you'll miss the # prefix.

-- TWiki:Main.PeterThoeny - 2023-01-01

Best to fix the $TWiki::regex{anchorRegex} instead of adding an exception in lib/TWiki/Render.pm

-- TWiki:Main.PeterThoeny - 2023-01-01

Is there a way that I can simply replicate this error as I have difficulty understanding it and I am not able to test it here.

I tried to replicate it by copying https://develop.twiki.org/do/view/Bugs/WebChanges and https://develop.twiki.org/do/view/Bugs/WebChanges?raw=expandvariables in our environment but it does not replicate the problem. I still have no idea where and how it cause the error.

-- TWiki:Main.CalvinSo - 2023-01-04

OK, I finally found out the cause, it is https://develop.twiki.org/do/view/Bugs/Item7921, let me dig it a little bit

-- TWiki:Main.CalvinSo - 2023-01-05

I think I grab what is going on,

I am reverting this line back to change the $regex{anchorRegex}

$text =~ s/$STARTWW(?:($TWiki::regex{webNameRegex})\.)?($TWiki::regex{wikiWordRegex}|$TWiki::regex{abbrevRegex})($TWiki::regex{anchorRegex})?/_handleWikiWord($this,$theWeb,$1,$2,$3)/geom;

$regex{anchorRegex} = qr/#(?:[$regex{mixedAlphaNum}_\/\?\-\+.\'=~:!\$&*,;]|(?:%[0-9a-fA-F]{2}))+(?<![\s*.,!?;:<|])/o;

-- TWiki:Main.CalvinSo - 2023-01-05

I see the anchorRegex is already enhanced to qr/#(?:[$regex{mixedAlphaNum}_\/\?\-\+.\'=~:!\$&*,;]|%[0-9a-fA-F]{2})+/o;.

So maybe revert lib/TWiki/Render.pm to this?

$text =~ s/$STARTWW(?:($TWiki::regex{webNameRegex})\.)?($TWiki::regex{wikiWordRegex}|$TWiki::regex{abbrevRegex})($TWiki::regex{anchorRegex})?/_handleWikiWord( $this,$theWeb,$1,$2,$3)/geom;

-- TWiki:Main.PeterThoeny - 2023-01-07

I was like that for a while but I remembered you said you want to ignore punctuation for autolink, so that was the reason why Render.pm was changed. As it generated some unknown issue, I am trying to move that part to anchorRegex.

Please let me know your idea.

-- TWiki:Main.CalvinSo - 2023-01-10

ItemTemplate
Summary	$regex{anchorRegex} not defined correctly
ReportedBy	TWiki:Main.CalvinSo
Codebase	6.1.0
SVN Range	TWiki-6.1.0-trunk, Mon, 05 Apr 2021, build 31021
AppliesTo	Engine
Component
Priority	Normal
CurrentState	Being Worked On
WaitingFor
Checkins	TWikirev:31058 TWikirev:31065
TargetRelease	patch
ReleasedIn	6.1.1

Item7937: $regex{anchorRegex} not defined correctly

Item Form Data

Edit Form Data

Detail