Sending code to REPL does not work for strings longer than 500 characters (OS X)

I'm falling into a strange behavior. The following chunk: https://gist.github.com/tofferPika/4c4cc8e004a38137e163 run fine from the terminal with the lua interpreter (5.2) but refuse with the lua-mode process. I can't figure where the code break the string passed to the eval process, I first thinked it was the nested varargs, or some function name identical to lisp one... Additionaly, when moving functions arround and add blank lines here and there, the error message change.
I hope it make sense

Thank you for the report, however, I can't reproduce the issue:

Could you fetch the current master and reproduce the issue using emacs -Q -l path/to/checkout/lua-mode.el?

I've this issue from the current master. Also I forgot to mention it is from OS X Emacs 24.4 (0.9). I've the same result when running with the -Q -l flags. There is a screenshot here : https://dl.dropboxusercontent.com/u/50413630/lua-mode-shot.png. Tell me how can I help you to find the problem, I'm not a "Lisper" but I should be able to read and understand the logic.

Here's where you have the complete command that is being sent to the subprocess and that has to be syntactically-correct Lua code:
https://github.com/immerrr/lua-mode/tree/master/lua-mode.el#L1789

You can insert (message "%s" command) at that line and check the output in the *Messages* buffer. I'd guess that something goes wrong in lua-make-lua-string function that should turn the source code being sent into a properly escaped Lua string. If you don't see anything obvious, put the string from *Messages* buffer here and I'll try to have a look at it.

Ok, after some tests, I found it is not a string escaping problem. It happen when there is more than 500 characters sent. I've checked process-send-string doc, and effectively it mention that if the string is more 500 characters long, it is sent in several bunches. But I stick with why only the first part is evaluated before the complete string is sent.

My workaround for this is to use my own function instead of lua-send-buffer which hits this problem:

(defun pnh-lua-send-file ()
  (interactive)
  (lua-send-string (format "_ = dofile('%s') print('ok')" buffer-file-name)))

While it would be great to fix this in lua-send-string, in the mean time changing lua-send-buffer to save and refer the Lua subprocess to the file on disk would solve 90% of the symptoms.

I can send a patch for this if you want.

This is an issue on Windows as well. I suggest to solve it in the way, similar to the how it is done in the SLIME (IDE for Common Lisp for Emacs) - save the chunk into a temporary file and then load it into the interpreter (dofile(...)).

I have redefined (lua-send-string ...) in my init file to the following code:

(defun lua-send-string (str)
  "Load STR plus a newline to Lua subprocess.

If `lua-process' is nil or dead, start a new process first."
  (let ((tmp-file (make-temp-file "luamode" nil ".tmp"))
        (lua-process (lua-get-create-process)))
    ;; write data into temporary file
    (with-temp-buffer
      (insert (if (string-equal (substring str -1) "\n")
                  str
                (concat str "\n")))
      (write-file tmp-file))
    ;; evaluate data in the temporary file and then remove it
    (process-send-string lua-process (format (concat
                                              "\nlocal luamode_tmpfile = '%s';"
                                              "_ = dofile(luamode_tmpfile);"
                                              "os.remove(luamode_tmpfile);\n") tmp-file))))

It seems to solve this issue.

P.S.
I could make a pull request if you want to. It seems technomancy suggested similar idea.

The same solution with much better error handling on the Lua side. Compared to the previous one it tries hard to remove the temporary file.

(defun lua-send-string (str)
  "Load STR plus a newline into Lua subprocess.

If `lua-process' is nil or dead, start a new process first."
  (let ((tmp-file (make-temp-file "luamode" nil ".tmp"))
        (lua-process (lua-get-create-process)))
    ;; write data into temporary file
    (with-temp-buffer
      (insert (if (string-equal (substring str -1) "\n")
                  str
                (concat str "\n")))
      (write-file tmp-file))
    ;; evaluate data in the temporary file and then remove it
    (process-send-string
     lua-process
     (format (concat
              "\n"
              "local tmp = '%s';"
              "local res, e = pcall(function () "
              "  local do_loadstring = loadstring or load;"
              ""
              "  local f, e = io.open(tmp, 'r');" ; open temporary file
              "  if e then "
              "    os.remove(tmp);"
              "    error(e);"
              "    return;"
              "  end "
              ""
              "  local cont, e = f:read('*all');" ; read all data
              "  if e then "
              "    os.remove(tmp);"
              "    error(e);"
              "    return;"
              "  end "
              ""
              "  f:close(); f = nil;" ; close and remove file
              "  os.remove(tmp);"
              ""
              "  local f, e = do_loadstring(cont);" ; handle chunk
              "  if e then "
              "    error(e);"
              "    return;"
              "  end "
              ""
              "  return f();" ; execute chunk
              "end);"
              "if e then _, _ = os.remove(tmp); error(e); end" ; handle error, if any
              "\n") tmp-file))))

Fix sending of long code strings to REPL using temporary files (issue: …

I have made the pull request based on the code above, but it sends short code chunks (<= 500 characters) directly to the REPL (as it was). It uses temporary files for bigger code chunks. This hybrid approach works flawlessly for me.

There is no reason to use temporary files, you can send the code delimited with a suitable token inline using process-send-string on the Emacs side and io.read() on the Lua side which will read until it hits the delimiter (which can even be overriden and work over sockets). I've been using that for months in my own lua-mode and it works perfectly. If I get some free time I'll send a pull request but given that my mode is completely different and primarily works over sockets, it may take a while. I switched away from immerrr's lua-mode since it's too complicated and has a lot of problems that I didn't think (at the time) were worth spending time to fix.

Navigation Menu

Repository navigation

Sending code to REPL does not work for strings longer than 500 characters (OS X) #89

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Sending code to REPL does not work for strings longer than 500 characters (OS X) #89

Description

Activity

immerrr commented on Apr 12, 2015

bonbek commented on Apr 13, 2015

immerrr commented on Apr 13, 2015

bonbek commented on Apr 14, 2015

technomancy commented on Jun 6, 2016

arbv commented on Jun 22, 2016

arbv commented on Jun 22, 2016

arbv commented on Jun 22, 2016

arbv commented on Jun 22, 2016

atomontage commented on Sep 22, 2019

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions