Dramatic Slowdown of LiveCode Create Native: Classic Mode

Ask for help
1

I use LiveCode Create Native (macOS) in Classic Mode only for now. I have a large stack which has slowly but steadily grown to 8 250 cards in 762 MB over the last 15 years or so (since Runtime Revolution). For the last 3 months, I have not changed my MacBook Pro computer, macOS, LiveCode version or any of my handlers in scripts.

Last week, all of a sudden, one of my handlers (function MakeCount, about 150 lines of code as shown attached below) which had been taking 2 to 4 seconds to complete became extremely slow to take 3 minutes. The last change of the handler was definitely more than 3 months ago. It uses several “repeat for each line tCd in tIndex” kind of loops, but never “repeat with i = 1 to XX” which would be disastrously slow.

So, my question is whether there is some kind of threshold of the number of cards or stack size in MB or something, over which some operations become extremely slow like a step function (reminiscent of OS’s virtual memory)??

function MakeCount
put “!HyperInfo” into tStack
put fld “Index” into tIndex
put fld “Contents” into tContents
put
fld “Script” of cd “{List:Others}” & return &
fld “Script” of cd “{List:Sci-Fi Central}” & return &
fld “Script” of cd “{List:Tubi Sci-Fi}” & return &
fld “Script” of cd “{List:Watch Now - Sci-Fi & Fantasy}” into tSciFi
put fld “Script” of cd “1996-10-10|Sunatori’s Coined Neologism/Néologisme” into tNeologism
put fld “Script” of cd “2016-10-10|Sunatori’s Coined Text Replacements” into tReplacements
put fld “Script” of cd “2017-06-09|Innovation150” into tInnovation
put fld “Script” of cd “Opinions/Des avis” into tOpinions
put fld “Script” of cd “Portfolio/Portefeuille” into tPortfolio
filter lines of tPortfolio with pattern “
  • ”

    • put 0 into tApple
    put 0 into tLiveCode
    repeat for each line tLine in tContents
    if (“~/Dropbox/” is in tLine) then
    put URL (“binfile:” & tLine) into tTxts
    if (tTxts <> “”) then
    if (“/bug_reporter_” is in tLine) then
    repeat for each line tLinf in tTxts
    if (“<span” & quote & “>” is in tLinf) then
    put tApple + 1 into tApple
    end if
    end repeat
    put “Apple bugs” & tab & StripEmpty(tApple) & return after tReturn
    else if (“/LiveCode Quality Control Center” is in tLine) then
    repeat for each line tLinf in tTxts
    if (“<a href=” & quote & “show_bug.cgi?id=” is in tLinf) then
    put tLiveCode + 1 into tLiveCode
    end if
    end repeat
    put (tLiveCode / 2) into tLiveCode
    put “LiveCode bugs” & tab & StripEmpty(tLiveCode) & return after tReturn
    end if
    end if
    end if
    end repeat
    put “SUM(Apple bugs + LiveCode bugs)” & tab & StripEmpty(tApple + tLiveCode) & return & return after tReturn

    put 0 into i
    set itemDel to quote
    repeat for each line tLine in tInnovation
    if (“$<H3 ID=” is in (“$” & tLine)) then
    put item 2 of tLine into tLinf
    if (tLinf <> “”) and (tLinf <> “XXX”) and ((“#” & tLinf & quote & “>”) is not in tPortfolio) then
    put “• Missing in Portfolio: #” & tLinf & return after tCheck
    end if
    end if
    if (“<IMG SRC=” & quote & “…/” is not in tLind) and (“<IMG SRC=” & quote & “…/” is in tLine) and (“_A” is not in tLine) then
    put i + 1 into i
    end if
    put tLine into tLind
    end repeat
    repeat for each line tLine in tPortfolio
    if (“
  • <A HREF=” is in tLine) and (“#” is in tLine) then
    set itemDel to quote
    put item 2 of tLine into tLinf
    set itemDel to “#”
    put item 2 of tLinf into tLinf
    if (tLinf <> “”) and (tLinf <> “XXX”) and ((“

    ”) is not in tInnovation) then
    put “• Missing in Innovation: ID=” & quote & tLinf & quote & return after tCheck
    end if
    end if
    end repeat
    put i into tTotal
    put “Innovation150” & tab & StripEmpty(i) & return after tReturn
    put tOpinions & return & tPortfolio into tInnovation

    put 0 into tPotential
    put 0 into tArts
    put 0 into tWonders
    put 0 into tConcepts
    repeat for each line tCd in tIndex
    put fld “Title” of cd tCd into tTitle
    put char 1 to 4 of tTitle into tYYYY
    put line 1 of fld “Contents” of cd tCd into tFirst
    if (tYYYY is an integer) and (“Sunatori’s” is in tTitle) then
    repeat for each line tContent in tContents
    if (“~/Dropbox/” is not in tContent) and (char 1 of tContent <> “-”) then
    if (“X=Potential” is in tFirst) then
    put tPotential + 1 into tPotential
    exit repeat
    else if (“X=Arts” is in tFirst) then
    put tArts + 1 into tArts
    exit repeat
    else if (“X=Wonders” is in tFirst) then
    put tWonders + 1 into tWonders
    exit repeat
    else if (“X=Concepts” is in tFirst) then
    put tConcepts + 1 into tConcepts
    exit repeat
    end if
    end if
    end repeat
    end if
    end repeat
    put
    “X=Potential” & tab & tPotential & return &
    “X=Arts” & tab & tArts & return &
    “X=Wonders” & tab & tWonders & return &
    “X=Concepts” & tab & tConcepts & return after tReturn
    put tTotal + tPotential + tArts + tWonders + tConcepts into tTotal
    put “SUM(Innovation150 + X=Potential + X=Arts + X=Wonders + X=Concepts)” & tab & StripEmpty(tTotal) & return & return after tReturn

    put 0 into i
    repeat for each line tLine in tReplacements
    if (“$
  • ” is in (“$” & tLine)) and ((“[” is not in tLine) and (“]” is not in tLine)) then
    put i + 1 into i
    end if
    end repeat
    put “Replacements” & tab & StripEmpty(i) & return after tReturn

    • put 0 into i
    repeat for each line tLine in tNeologism
    if (“$
  • ” is in (“$” & tLine)) and ((“[” is not in tLine) and (“]” is not in tLine)) then
    put i + 1 into i
    end if
    end repeat
    put “Neologism” & tab & StripEmpty(i) & return after tReturn

    • put 0 into i
    set itemDel to tab
    repeat for each line tLine in tSciFi
    put item 1 of tLine into tLine1
    put item 2 of tLine into tLine2
    if (“A” is in tLine2) or (“B” is in tLine2) or (“C” is in tLine2) then
    if (tLine1 is not in tText) then
    put i + 1 into i
    end if
    put tLine & return after tText
    end if
    end repeat
    put “Sci-Fi” & tab & StripEmpty(i) & return after tReturn

    put 0 into i
    repeat for each line tLine in tIndex
    if (“§” is not in tLine) and (“|” is in tLine) and (char 1 to 4 of tLine is an integer) then
    put i + 1 into i
    end if
    end repeat
    put “Weblog” & tab & StripEmpty(i) & return after tReturn

    if (gDoIt = true) and (tCheck <> “”) then
    answer tCheck – as sheet
    end if
    return tReturn
    end MakeCount

    •
    2

    Hi @GS.Sunatori thanks for the post!

    It’s possible that a unicode character has slipped into one of the fields. You can experiment using:

    put textDecode(textEncode(<text>, "native"), "native") into <var>

    for the fields you’re copying into variables and seeing which one speeds it up again. The above will turn a unicode string into a native string, replacing unicode characters with ‘?’

    3

    Hej Adam,

    You are my lifesaver!!! The extreme slowdown was indeed caused by several Unicode characters that had slipped into some of the fields!!

    In order to detect where these Unicode characters are, I coded a mouseUp handler with the following line that you gave me.

    
put textDecode(textEncode(<text>, "native"), "native") into <var>

    Out of 8 256 cards, there were various Unicode characters (smileys) in text fields of 28 cards, due to Copy & Paste of webpage text from Safari.app (macOS) browser. The characters were replaced with “?” by your textDecode(textEncode()) code. One of the fields has 794 851 chars, so the task was tedious, but I finally managed to manually eliminate the Unicode characters (smileys)!

    Now, my function MakeCount runs in 3 seconds instead of 3 minutes!!

    Furthermore, I wrote a routine that checks for the existence of Unicode char every time text in a field is changed.

    This Unicode oddity should be documented somewhere. Do you happen to know if there is any plan to make Unicode text run almost as fast as ASCII text?

    Thanks again, Adam!!!