emacs-elpa-diffs
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[elpa] externals/phps-mode 800ba6e: Improved indentation support


From: Christian Johansson
Subject: [elpa] externals/phps-mode 800ba6e: Improved indentation support
Date: Fri, 16 Aug 2019 01:07:31 -0400 (EDT)

branch: externals/phps-mode
commit 800ba6ea9573eb924981e2ccc5c9f743ff5c7a81
Author: Christian Johansson <address@hidden>
Commit: Christian Johansson <address@hidden>

    Improved indentation support
---
 Makefile                                           |   2 +-
 README.md                                          |   4 +-
 docs/heuristics.md                                 |   8 +-
 docs/todo.md                                       |   4 +-
 phps-mode-automation.el                            |  35 +--
 phps-mode-functions.el                             | 235 ++++++++++++++-------
 phps-mode-lexer.el                                 |  94 ++++++---
 phps-mode-semantic.el                              |   1 +
 phps-mode-tags.el                                  |  57 +----
 phps-mode-test-functions.el                        | 101 +++++++--
 phps-mode-test-integration.el                      |  32 ++-
 phps-mode-test-lexer.el                            |  35 +--
 phps-mode-test-syntax-table.el                     |  10 +-
 phps-mode-test.el                                  |  31 +--
 phps-mode.el                                       | 102 +++++++--
 sample-php-files/alternative-control-structure.php |   5 +
 sample-php-files/functions.php                     |  17 ++
 17 files changed, 500 insertions(+), 273 deletions(-)

diff --git a/Makefile b/Makefile
index 05a2472..6a0d0c9 100644
--- a/Makefile
+++ b/Makefile
@@ -4,7 +4,7 @@ ifdef emacs
 endif
 EMACS_CMD := $(EMACS) -Q -batch -L .
 
-EL  := phps-mode-flycheck.el phps-mode-flymake.el phps-mode-font-lock.el 
phps-mode-functions.el phps-mode-lexer.el phps-mode-map.el 
phps-mode-semantic.el phps-mode-syntax-table.el phps-mode-test.el 
phps-mode-test-functions.el phps-mode-test-integration.el 
phps-mode-test-lexer.el phps-mode-test-parser.el phps-mode-test-syntax-table.el 
phps-mode.el
+EL  := phps-mode-automation.el phps-mode-flymake.el phps-mode-functions.el 
phps-mode-lexer.el phps-mode-semantic.el phps-mode-syntax-table.el 
phps-mode-tags.el phps-mode-test-functions.el phps-mode-test-integration.el 
phps-mode-test-lexer.el phps-mode-test-parser.el phps-mode-test-syntax-table.el 
phps-mode-test.el phps-mode.el
 ELC := $(EL:.el=.elc)
 
 .PHONY: clean
diff --git a/README.md b/README.md
index d78f641..b8dda02 100644
--- a/README.md
+++ b/README.md
@@ -22,7 +22,7 @@ This mode does not require PHP installed on your computer 
because it has a built
 * Minimal mode map
 * Tested using unit tests and integration tests
 * Travis support
-* Included in GNU ELPA package archives
+* Included in GNU ELPA package archive
 
 
 ## Develop
@@ -105,7 +105,7 @@ If you have downloaded manually i.e. to 
`~/.emacs.d/phps-mode/` you need to add
 
 ### Install via package manager
 
-You can install via ELPA (`M-x package-install` `phps-mode`), package will be 
loaded automatically then.
+You can install via ELPA (`M-x package-install` + `RET` + `phps-mode` + 
`RET`), package will now be loaded automatically when Emacs starts.
 
 ### Install, load and configure via use-package
 
diff --git a/docs/heuristics.md b/docs/heuristics.md
index 744cfbe..3958a31 100644
--- a/docs/heuristics.md
+++ b/docs/heuristics.md
@@ -1,8 +1,10 @@
 ## Heuristics
 
-These should solve the problem of freezing editor when making white-space 
changes to code.
+These should solve the problem of freezing editor when making white-space 
changes to code. Otherwise a full incremental re-parse would be triggered more 
often than necessary.
 
-* When pressing return when line after cursor is only white-space
-* When pressing backspace when line before cursor is only white-space
+* When pressing return when the rest of the current line after cursor is only 
white-space
+* When pressing backspace when the rest of the current line before cursor is 
only white-space
+
+In both of these cases, move indexes of tokens, states, indentations and imenu
 
 [Back to start](../../../)
diff --git a/docs/todo.md b/docs/todo.md
index ab92428..4c07a64 100644
--- a/docs/todo.md
+++ b/docs/todo.md
@@ -3,10 +3,10 @@
 *With current progress estimates:*
 
 * Add to MELPA package archive (50%)
-* A set of heuristics to improve large-file incremental change handling (50%)
+* A set of heuristics to improve large-file incremental change handling (75%)
 * Wisent LALR parser based on official PHP yacc parser automatically converted 
grammar (50%)
+* mmm-mode support (50%)
 * Full integration with Emacs Semantic subsystem (30%)
-* Approach flycheck about including support for this module by default (0%)
 * Support indentation for HTML/XML, Javascript and CSS content inside 
inline_html content (0%)
 * Eldoc support (0%)
 * Flymake support (0%)
diff --git a/phps-mode-automation.el b/phps-mode-automation.el
index 980f460..3ccc5ef 100644
--- a/phps-mode-automation.el
+++ b/phps-mode-automation.el
@@ -24,26 +24,29 @@
 
 ;;; AST should be like this: (root (block (rule (logic))))
 
+;;; This entire file is WIP / TODO
+
 
 ;;; Code:
 
-(add-to-list 'load-path (expand-file-name (concat user-emacs-directory 
"emacs-wisent-grammar-converter/")))
-(autoload 'emacs-wisent-grammar-converter/generate-grammar-from-filename 
"emacs-wisent-grammar-converter")
+;; (require 'emacs-wisent-grammar-converter)
+
+;; (add-to-list 'load-path (expand-file-name (concat user-emacs-directory 
"emacs-wisent-grammar-converter/")))
 
-(let ((php-yacc-url 
"https://raw.githubusercontent.com/php/php-src/master/Zend/zend_language_parser.y";)
-      (php-yacc-file (expand-file-name "zend_language_parser.y"))
-      (wisent-destination (expand-file-name "zend_language_parser.wy")))
+;; (let ((php-yacc-url 
"https://raw.githubusercontent.com/php/php-src/master/Zend/zend_language_parser.y";)
+;;       (php-yacc-file (expand-file-name "zend_language_parser.y"))
+;;       (wisent-destination (expand-file-name "zend_language_parser.wy")))
 
-  ;; Download Yacc if not available
-  (unless (file-exists-p php-yacc-file)
-    (message "Downloading PHP Yacc grammar..")
-    (url-copy-file php-yacc-url php-yacc-file t t)
-    (message "Download completed"))
+;;   ;; Download Yacc if not available
+;;   (unless (file-exists-p php-yacc-file)
+;;     (message "Downloading PHP Yacc grammar..")
+;;     (url-copy-file php-yacc-url php-yacc-file t t)
+;;     (message "Download completed"))
 
-  ;; Generate grammar
-  (message "Generating Wisent grammar..")
-  (emacs-wisent-grammar-converter/generate-grammar-from-filename php-yacc-file 
wisent-destination)
-  (message "Automation completed"))
+;;   ;; Generate grammar
+;;   (message "Generating Wisent grammar..")
+;;   (emacs-wisent-grammar-converter/generate-grammar-from-filename 
php-yacc-file wisent-destination)
+;;   (message "Automation completed"))
 
-(provide 'phps-automation)
-;;; phps-automation.el ends here
+(provide 'phps-mode-automation)
+;;; phps-mode-automation.el ends here
diff --git a/phps-mode-functions.el b/phps-mode-functions.el
index 5e6e920..09bce05 100644
--- a/phps-mode-functions.el
+++ b/phps-mode-functions.el
@@ -23,9 +23,7 @@
 
 ;;; Code:
 
-(autoload 'phps-mode-lexer-run-incremental "phps-mode-lexer")
-(autoload 'phps-mode-lexer-move-tokens "phps-mode-lexer")
-(autoload 'phps-mode-lexer-move-states "phps-mode-lexer")
+(require 'phps-mode-lexer)
 
 (defvar phps-mode-functions-allow-after-change t
   "Flag to tell us whether after change detection is enabled or not.")
@@ -92,10 +90,17 @@
           (setq line-indent (gethash line-number old-lines-indents))))
       lines-indents)))
 
+(defun phps-mode-functions-move-imenu-index (start diff)
+  "Moved imenu from START by DIFF points."
+  (when phps-mode-functions-imenu
+    (setq phps-mode-functions-imenu (phps-mode-functions-get-moved-imenu 
phps-mode-functions-imenu start diff))))
+
 (defun phps-mode-functions-move-lines-indent (start-line-number diff)
   "Move lines indent from START-LINE-NUMBER with DIFF points."
-  (setq phps-mode-functions-lines-indent 
(phps-mode-functions-get-moved-lines-indent phps-mode-functions-lines-indent 
start-line-number diff)))
-
+  (when phps-mode-functions-lines-indent
+    ;; (message "Moving line-indent index from %s with %s" start-line-number 
diff)
+    (setq phps-mode-functions-lines-indent 
(phps-mode-functions-get-moved-lines-indent phps-mode-functions-lines-indent 
start-line-number diff))))
+  
 (defun phps-mode-functions-get-lines-indent ()
   "Return lines indent, process buffer if not done already."
   (phps-mode-functions-process-current-buffer)
@@ -106,6 +111,29 @@
   (phps-mode-functions-process-current-buffer)
   phps-mode-functions-imenu)
 
+(defun phps-mode-functions-get-moved-imenu (old-index start diff)
+  "Move imenu-index OLD-INDEX beginning from START with DIFF."
+  (let ((new-index '()))
+
+    (when old-index
+      (if (and (listp old-index)
+               (listp (car old-index)))
+          (dolist (item old-index)
+            (let ((sub-item (phps-mode-functions-get-moved-imenu item start 
diff)))
+              (push (car sub-item) new-index)))
+        (let ((item old-index))
+          (let ((item-label (car item)))
+            (if (listp (cdr item))
+                (let ((sub-item (phps-mode-functions-get-moved-imenu (cdr 
item) start diff)))
+                  (push `(,item-label . ,(nreverse sub-item)) new-index))
+              (let ((item-start (cdr item)))
+                (when (>= item-start start)
+                  (setq item-start (+ item-start diff)))
+                (push `(,item-label . ,item-start) new-index))))
+          )))
+
+    new-index))
+
 (defun phps-mode-functions--get-lines-in-buffer (beg end)
   "Return the number of lines in buffer between BEG and END."
   (phps-mode-functions--get-lines-in-string (buffer-substring-no-properties 
beg end)))
@@ -163,11 +191,17 @@
               (in-assignment-round-bracket-level nil)
               (in-assignment-square-bracket-level nil)
               (in-assignment-level 0)
+              (in-object-operator nil)
+              (in-object-operator-round-bracket-level nil)
+              (in-object-operator-square-bracket-level nil)
+              (after-object-operator nil)
+              (in-object-operator-level 0)
               (in-class-declaration nil)
               (in-class-declaration-level 0)
               (in-return nil)
               (in-return-curly-bracket-level nil)
               (in-return-level 0)
+              (previous-token nil)
               (token nil)
               (token-start nil)
               (token-end nil)
@@ -208,6 +242,7 @@
                   (next-token-end-line-number nil))
 
               (when token
+                ;; NOTE We use a incremental-line-number calculation because 
`line-at-pos' takes a lot of time
                 (setq incremental-line-number (+ incremental-line-number 
(phps-mode-functions--get-lines-in-buffer token-end next-token-start))))
 
               ;; Handle the pseudo-token for last-line
@@ -216,12 +251,17 @@
                     (setq next-token-start-line-number (1+ 
token-start-line-number))
                     (setq next-token-end-line-number (1+ 
token-end-line-number)))
                 (setq next-token-start-line-number incremental-line-number)
+
+                ;; NOTE We use a incremental-line-number calculation because 
`line-at-pos' takes a lot of time
                 (setq incremental-line-number (+ incremental-line-number 
(phps-mode-functions--get-lines-in-buffer next-token-start next-token-end)))
                 (setq next-token-end-line-number incremental-line-number)
                 (when phps-mode-functions-verbose
                   (message "Token '%s' pos: %s-%s lines: %s-%s" next-token 
next-token-start next-token-end next-token-start-line-number 
next-token-end-line-number)))
 
-              ;; Token logic - we have one token look-ahead at this point
+              ;; Token logic - we have one-two token look-ahead at this point
+              ;; `token' is previous token
+              ;; `next-token' is current token
+              ;; `previous-token' is maybe two tokens back
               (when token
 
 
@@ -307,15 +347,18 @@
 
                  (t (cond
 
-                     ((equal token 'T_NAMESPACE)
+                     ((and (not imenu-in-namespace-name)
+                           (equal token 'T_NAMESPACE))
                       (setq imenu-in-namespace-name nil)
                       (setq imenu-in-namespace-declaration t))
 
-                     ((equal token 'T_CLASS)
+                     ((and (not imenu-in-class-name)
+                           (equal token 'T_CLASS))
                       (setq imenu-in-class-name nil)
                       (setq imenu-in-class-declaration t))
 
-                     ((equal token 'T_FUNCTION)
+                     ((and (not imenu-in-function-name)
+                           (equal token 'T_FUNCTION))
                       (setq imenu-in-function-name nil)
                       (setq imenu-in-function-declaration t)))))
 
@@ -350,7 +393,7 @@
                           (setq in-class-declaration nil)
                           (setq in-class-declaration-level 0)
 
-                          (when (not class-declaration-started-this-line)
+                          (unless class-declaration-started-this-line
                             (setq column-level (1- column-level))
                             (pop nesting-stack))
 
@@ -360,7 +403,11 @@
                           )
                       (when first-token-on-line
                         (setq in-class-declaration-level 1)))
-                  (when (equal token 'T_CLASS)
+
+                  ;; If ::class is used as a magical class constant it should 
not be considered start of a class declaration
+                  (when (and (equal token 'T_CLASS)
+                             (or (not previous-token)
+                                 (not (equal previous-token 
'T_PAAMAYIM_NEKUDOTAYIM))))
                     (setq in-class-declaration t)
                     (setq in-class-declaration-level 1)
                     (setq class-declaration-started-this-line t)))
@@ -452,7 +499,7 @@
                             )
 
                         ;; Don't start inline control structures after a while 
($condition); expression
-                        (when (not (string= token ";"))
+                        (unless (string= token ";")
                           (when phps-mode-functions-verbose
                             (message "\nStarted inline control-structure after 
%s at %s\n" after-special-control-structure-token token))
 
@@ -547,24 +594,50 @@
                                  (or (< round-bracket-level (car 
in-assignment-round-bracket-level))
                                      (and
                                       (= round-bracket-level (car 
in-assignment-round-bracket-level))
-                                      (string= next-token ")"))))
+                                      (= square-bracket-level (car 
in-assignment-square-bracket-level))
+                                      (or (string= next-token ")")
+                                          (string= next-token "]")))))
                             (and (string= token ",")
                                  (= round-bracket-level (car 
in-assignment-round-bracket-level))
                                  (= square-bracket-level (car 
in-assignment-square-bracket-level)))
-                            (and (string= token"]")
-                                 (< square-bracket-level (car 
in-assignment-square-bracket-level)))
+                            (and (string= token "]")
+                                 (or (< square-bracket-level (car 
in-assignment-square-bracket-level))
+                                     (and
+                                      (= square-bracket-level (car 
in-assignment-square-bracket-level))
+                                      (= round-bracket-level (car 
in-assignment-round-bracket-level))
+                                      (or (string= next-token "]")
+                                          (string= next-token ")")))))
                             (and (equal token 'T_FUNCTION)
                                  (= round-bracket-level (car 
in-assignment-round-bracket-level))))
 
-                    ;; NOTE Ending an assignment because of function token is 
to support PSR-2 Closures
+                    ;; NOTE Ending an assignment because of a T_FUNCTION token 
is to support PSR-2 Closures
                     
                     (when phps-mode-functions-verbose
-                      (message "Ended assignment at %s %s" token next-token))
+                      (message "Ended assignment %s at %s %s" 
in-assignment-level token next-token))
                     (pop in-assignment-square-bracket-level)
                     (pop in-assignment-round-bracket-level)
                     (unless in-assignment-round-bracket-level
                       (setq in-assignment nil))
-                    (setq in-assignment-level (1- in-assignment-level))))
+                    (setq in-assignment-level (1- in-assignment-level))
+
+                    ;; Did we end two assignment at once?
+                    (when (and
+                           in-assignment-round-bracket-level
+                           in-assignment-square-bracket-level
+                           (= round-bracket-level (car 
in-assignment-round-bracket-level))
+                           (= square-bracket-level (car 
in-assignment-square-bracket-level))
+                           (or (string= next-token ")")
+                               (string= next-token "]")))
+                      (when phps-mode-functions-verbose
+                        (message "Ended another assignment %s at %s %s" 
in-assignment-level token next-token))
+                      (pop in-assignment-square-bracket-level)
+                      (pop in-assignment-round-bracket-level)
+                      (unless in-assignment-round-bracket-level
+                        (setq in-assignment nil))
+                      (setq in-assignment-level (1- in-assignment-level)))
+
+                    ))
+
                 (when (and (not after-special-control-structure)
                            (or (string= token "=")
                                (equal token 'T_DOUBLE_ARROW)
@@ -588,6 +661,45 @@
                   (push square-bracket-level 
in-assignment-square-bracket-level)
                   (setq in-assignment-level (1+ in-assignment-level)))
 
+                ;; Second token after a object-operator
+                (when (and
+                       in-object-operator
+                       in-object-operator-round-bracket-level
+                       in-object-operator-square-bracket-level
+                       (<= round-bracket-level (car 
in-object-operator-round-bracket-level))
+                       (<= square-bracket-level (car 
in-object-operator-square-bracket-level))
+                       (not (or
+                             (equal next-token 'T_OBJECT_OPERATOR)
+                             (equal next-token 'T_PAAMAYIM_NEKUDOTAYIM))))
+                  (when phps-mode-functions-verbose
+                    (message "Ended object-operator at %s %s at level %s" 
token next-token in-object-operator-level))
+                  (pop in-object-operator-round-bracket-level)
+                  (pop in-object-operator-square-bracket-level)
+                  (setq in-object-operator-level (1- in-object-operator-level))
+                  (when (= in-object-operator-level 0)
+                    (setq in-object-operator nil)))
+
+                ;; First token after a object-operator
+                (when after-object-operator
+                  (when (or (equal next-token 'T_STRING)
+                            (string= next-token "("))
+                    (progn
+                      (when phps-mode-functions-verbose
+                        (message "Started object-operator at %s %s on level 
%s"  token next-token in-object-operator-level))
+                      (push round-bracket-level 
in-object-operator-round-bracket-level)
+                      (push square-bracket-level 
in-object-operator-square-bracket-level)
+                      (setq in-object-operator t)
+                      (setq in-object-operator-level (1+ 
in-object-operator-level))))
+                  (setq after-object-operator nil))
+
+                ;; Starting object-operator?
+                (when (and (or (equal token 'T_OBJECT_OPERATOR)
+                               (equal token 'T_PAAMAYIM_NEKUDOTAYIM))
+                           (equal next-token 'T_STRING))
+                  (when phps-mode-functions-verbose
+                    (message "After object-operator at %s level %s"  token 
in-object-operator-level))
+                  (setq after-object-operator t))
+
                 ;; Keep track of return expressions
                 (when in-return
                   (when (and (string= token ";")
@@ -606,10 +718,6 @@
                   (push curly-bracket-level in-return-curly-bracket-level)
                   (setq in-return-level (1+ in-return-level)))
 
-                ;; Keep track of object operators
-                (when (and (equal token 'T_OBJECT_OPERATOR)
-                           first-token-on-line))
-
                 ;; Did we encounter a token that supports extra special 
alternative control structures?
                 (when (equal token 'T_CASE)
                   (setq after-extra-special-control-structure t)
@@ -635,7 +743,7 @@
                   (message "Processing token: %s" token))
                 
                 ;; Calculate nesting
-                (setq nesting-end (+ round-bracket-level square-bracket-level 
curly-bracket-level alternative-control-structure-level in-assignment-level 
in-class-declaration-level in-concatenation-level in-return-level))
+                (setq nesting-end (+ round-bracket-level square-bracket-level 
curly-bracket-level alternative-control-structure-level in-assignment-level 
in-class-declaration-level in-concatenation-level in-return-level 
in-object-operator-level))
 
                 ;; Keep track of whether we are inside a HEREDOC or NOWDOC
                 (when (equal token 'T_START_HEREDOC)
@@ -805,7 +913,7 @@
 
 
                       ;; Calculate indentation level at start of line
-                      (setq nesting-start (+ round-bracket-level 
square-bracket-level curly-bracket-level alternative-control-structure-level 
in-assignment-level in-class-declaration-level in-concatenation-level 
in-return-level))
+                      (setq nesting-start (+ round-bracket-level 
square-bracket-level curly-bracket-level alternative-control-structure-level 
in-assignment-level in-class-declaration-level in-concatenation-level 
in-return-level in-object-operator-level))
 
                       ;; Set initial values for tracking first token
                       (when (> token-start-line-number last-line-number)
@@ -818,8 +926,8 @@
                         (setq special-control-structure-started-this-line 
nil)))
 
                   ;; Current token is not first if it's not <?php or <?=
-                  (when (not (or (equal token 'T_OPEN_TAG)
-                                 (equal token 'T_OPEN_TAG_WITH_ECHO)))
+                  (unless (or (equal token 'T_OPEN_TAG)
+                              (equal token 'T_OPEN_TAG_WITH_ECHO))
                     (setq first-token-on-line nil))
 
                   (when (> token-end-line-number token-start-line-number)
@@ -834,6 +942,7 @@
                     (setq tuning-level 0))))
 
               ;; Update current token
+              (setq previous-token token)
               (setq token next-token)
               (setq token-start next-token-start)
               (setq token-end next-token-end)
@@ -851,7 +960,6 @@
       (progn
         ;; (message "Running advice")
         (let ((old-pos (point))
-              (new-pos)
               (looking-at-whitespace (looking-at-p "[\ \n\t\r]*\n"))
               (old-line-number (line-number-at-pos)))
 
@@ -860,19 +968,20 @@
                 ;; (message "Looking at white-space")
 
                 ;; Temporarily disable change detection to not trigger 
incremental lexer
+
+                ;; We move indexes before calling old-function
+                ;; because old-function could be `newline-and-indent'
+                ;; and this would trigger `indent-line'
+                ;; which will trigger processing buffer
+                (phps-mode-lexer-move-tokens old-pos 1)
+                (phps-mode-lexer-move-states old-pos 1)
+                (phps-mode-functions-move-imenu-index old-pos 1)
+                (phps-mode-functions-move-lines-indent old-line-number 1)
+
                 (setq phps-mode-functions-allow-after-change nil)
                 (apply old-function arguments)
-                (setq phps-mode-functions-allow-after-change t)
-                
-                (setq new-pos (point))
-                (let ((diff (- new-pos old-pos)))
-                  (when (> diff 0)
-                    (phps-mode-lexer-move-tokens old-pos diff)
-                    (phps-mode-lexer-move-states old-pos diff)
-                    (phps-mode-functions-move-lines-indent old-line-number 1)
-                    ;; TODO Move imenu-index?
-                    ;; (message "Old pos %s, new pos: %s, diff: %s" old-pos 
new-pos diff)
-                    )))
+                (setq phps-mode-functions-allow-after-change t))
+
             (apply old-function arguments)
             ;; (message "Not looking at white-space")
             )))
@@ -889,7 +998,7 @@
               (current-indentation (current-indentation))
               (line-start (line-beginning-position)))
 
-          (when (null current-indentation)
+          (unless current-indentation
             (setq current-indentation 0))
 
           ;; Only continue if current indentation is wrong
@@ -900,13 +1009,20 @@
 
               (indent-line-to indent-sum)
 
+
               ;; When indent is changed the trailing tokens and states just 
need to adjust their positions, this will improve speed of indent-region a lot
               (phps-mode-lexer-move-tokens line-start indent-diff)
               (phps-mode-lexer-move-states line-start indent-diff)
+              (phps-mode-functions-move-imenu-index line-start indent-diff)
+
+              ;; (message "Diff after indent at %s is %s" line-start 
indent-diff)
 
               ;; Reset change flag
-              (phps-mode-functions-reset-buffer-changes-start))))))))
+              (phps-mode-functions-reset-buffer-changes-start)
+
+              )))))))
 
+;; TODO Consider how imenu-index should be affected by this
 (defun phps-mode-functions-after-change (start _stop _length)
   "Track buffer change from START to STOP with length LENGTH."
   (when (and (string= major-mode "phps-mode")
@@ -1021,45 +1137,6 @@
             (line-move 1))
           (setq current-line-number (1+ current-line-number)))))))
 
-(defun phps-mode-functions-init ()
-  "PHP specific init-cleanup routines."
-
-  ;; Custom indentation
-  ;; NOTE Indent-region will call this on each line of region
-  (set (make-local-variable 'indent-line-function) 
#'phps-mode-functions-indent-line)
-
-  ;; Custom Imenu
-  (set (make-local-variable 'imenu-create-index-function) 
#'phps-mode-functions-imenu-create-index)
-
-  ;; Should we follow PSR-2?
-  (when (and (boundp 'phps-mode-use-psr-2)
-             phps-mode-use-psr-2)
-
-    ;; Code MUST use an indent of 4 spaces
-    (set (make-local-variable 'tab-width) 4)
-
-    ;; MUST NOT use tabs for indenting
-    (set (make-local-variable 'indent-tabs-mode) nil))
-
-  ;; Add support for moving indexes quickly when making newlines
-  (advice-add #'newline :around #'phps-mode-functions-around-newline)
-
-  ;; Reset flags
-  (set (make-local-variable 'phps-mode-functions-allow-after-change) t)
-  (set (make-local-variable 'phps-mode-functions-buffer-changes-start) nil)
-  (set (make-local-variable 'phps-mode-functions-lines-indent) nil)
-  (set (make-local-variable 'phps-mode-functions-imenu) nil)
-  (set (make-local-variable 'phps-mode-functions-processed-buffer) nil)
-
-  ;; Make (comment-region) and (uncomment-region) work
-  (set (make-local-variable 'comment-region-function) 
#'phps-mode-functions-comment-region)
-  (set (make-local-variable 'uncomment-region-function) 
#'phps-mode-functions-uncomment-region)
-  (set (make-local-variable 'comment-start) "// ")
-  (set (make-local-variable 'comment-end) "")
-
-  ;; Support for change detection
-  (add-hook 'after-change-functions #'phps-mode-functions-after-change))
-
 (provide 'phps-mode-functions)
 
 ;;; phps-mode-functions.el ends here
diff --git a/phps-mode-lexer.el b/phps-mode-lexer.el
index 8ba81d5..eba8222 100644
--- a/phps-mode-lexer.el
+++ b/phps-mode-lexer.el
@@ -29,16 +29,12 @@
 
 ;;; Code:
 
+;; NOTE We use autoload here to circumvent recursive require
 (autoload 'phps-mode-functions-get-buffer-changes-start "phps-mode-functions")
 (autoload 'phps-mode-functions-reset-buffer-changes-start 
"phps-mode-functions")
 
-(autoload 'semantic-lex-reset-functions "semantic")
-(autoload 'define-lex "semantic/lex")
-(autoload 'semantic-lex "semantic/lex")
-(autoload 'semantic-lex-buffer "semantic/lex")
-(autoload 'semantic-lex-token "semantic/lex")
-(autoload 'semantic-lex-push-token "semantic/lex")
-(autoload 'define-lex-analyzer "semantic/lex")
+(require 'semantic)
+(require 'semantic/lex)
 
 ;; NOTE This line is required to pass byte-compilation
 (require 'semantic/wisent)
@@ -214,7 +210,20 @@
     (overlay-put (make-overlay start end) 'font-lock-face 
'font-lock-variable-name-face))
 
    ((string= token 'T_INLINE_HTML)
-    (overlay-put (make-overlay start end) 'font-lock-face 
'font-lock-comment-delimiter-face))
+
+    (overlay-put (make-overlay start end) 'font-lock-face 
'font-lock-comment-delimiter-face)
+
+    ;; Optional support for mmm-mode below
+    (if (and (boundp 'phps-mode-inline-mmm-submode)
+             phps-mode-inline-mmm-submode
+             (fboundp 'mmm-make-region))
+        (progn
+          ;; (message "Added mmm-submode '%s' from %s - %s" 
phps-mode-inline-mmm-submode start end)
+          (dolist (overlay (overlays-in start end))
+            (delete-overlay overlay))
+          ;; (mmm-make-region phps-mode-inline-mmm-submode start end)
+          )
+      ))
 
    ((string= token 'T_COMMENT)
     (overlay-put (make-overlay start end) 'font-lock-face 
'font-lock-comment-face))
@@ -1348,32 +1357,42 @@
      (lambda()
        (let* ((start (match-beginning 0))
               (end (match-end 0))
-              (_data (buffer-substring-no-properties start end)))
+              (_data (buffer-substring-no-properties start end))
+              (open-quote t))
+
+         ;; Move forward from the double-quote
          (forward-char)
-         ;; Handle the "" case
-         (if (looking-at-p "\"")
-             (progn
-               ;; (message "Empty double quoted string from %s to %s" start (+ 
start 2))
-               (phps-mode-lexer-RETURN_TOKEN 'T_CONSTANT_ENCAPSED_STRING start 
(+ start 2))
-               (forward-char))
+
+         (while open-quote
            (let ((string-start (search-forward-regexp (concat
-                                                       "\\([^\\\\]\""
+                                                       "\\(\""
                                                        "\\|\\$" 
phps-mode-lexer-LABEL
                                                        "\\|\\${" 
phps-mode-lexer-LABEL
                                                        "\\|{\\$" 
phps-mode-lexer-LABEL "\\)")
                                                       nil t)))
+
              ;; Do we find a ending double quote or starting variable?
              (if string-start
-                 (let ((string-start (match-beginning 0)))
-                   ;; (message "Double quoted string %s" double-quoted-string)
+                 (let ((string-start (match-beginning 0))
+                       (is-escaped nil))
+
+                   ;; Go to character before match start
+                   (goto-char (1- string-start))
+
+                   ;; Store whether character is escaped or not
+                   (setq is-escaped (looking-at-p "\\\\"))
+
                    ;; Do we find variable inside quote?
                    (goto-char string-start)
-                   (if (looking-at "[^\\]\"")
-                       (progn
-                         (let ((_double-quoted-string 
(buffer-substring-no-properties start (+ string-start 2))))
+
+                   ;; Process character if it's not escaped
+                   (if is-escaped
+                       (forward-char 2)
+                     (setq open-quote nil)
+                     (if (looking-at "\"")
+                         (let ((_double-quoted-string 
(buffer-substring-no-properties start (+ string-start 1))))
                            ;; (message "Double quoted string: %s" 
_double-quoted-string)
-                           (phps-mode-lexer-RETURN_TOKEN 
'T_CONSTANT_ENCAPSED_STRING start (+ string-start 2))))
-                     (progn
+                           (phps-mode-lexer-RETURN_TOKEN 
'T_CONSTANT_ENCAPSED_STRING start (+ string-start 1)))
                        ;; (message "Found variable after '%s'" 
(buffer-substring-no-properties start string-start))
                        (phps-mode-lexer-BEGIN phps-mode-lexer-ST_DOUBLE_QUOTES)
                        (phps-mode-lexer-RETURN_TOKEN "\"" start (1+ start))
@@ -1381,7 +1400,8 @@
                (progn
                  ;; (message "Found no ending quote, skipping to end")
                  (phps-mode-lexer-RETURN_TOKEN 'T_ERROR start (point-max))
-                 (phps-mode-lexer-MOVE_FORWARD (point-max)))))))))
+                 (phps-mode-lexer-MOVE_FORWARD (point-max))
+                 (setq open-quote nil))))))))
 
     (phps-mode-lexer-re2c-rule
      (and ST_IN_SCRIPTING (looking-at (concat "<<<" 
phps-mode-lexer-TABS_AND_SPACES "\\(" phps-mode-lexer-LABEL "\\|'" 
phps-mode-lexer-LABEL "'\\|\"" phps-mode-lexer-LABEL "\"\\)" 
phps-mode-lexer-NEWLINE)))
@@ -1600,7 +1620,8 @@
 
 (defun phps-mode-lexer-move-states (start diff)
   "Move lexer states after (or equal to) START with modification DIFF."
-  (setq phps-mode-lexer-states (phps-mode-lexer-get-moved-states 
phps-mode-lexer-states start diff)))
+  (when phps-mode-lexer-states
+    (setq phps-mode-lexer-states (phps-mode-lexer-get-moved-states 
phps-mode-lexer-states start diff))))
 
 (defun phps-mode-lexer-get-moved-states (states start diff)
   "Return moved lexer STATES after (or equal to) START with modification DIFF."
@@ -1615,16 +1636,20 @@
               (state-symbol (nth 2 state-object))
               (state-stack (nth 3 state-object)))
           (if (>= state-start start)
-            (let ((new-state-start (+ state-start diff))
-                  (new-state-end (+ state-end diff)))
-              (push (list new-state-start new-state-end state-symbol 
state-stack) new-states))
-            (push state-object new-states)))))
+              (let ((new-state-start (+ state-start diff))
+                    (new-state-end (+ state-end diff)))
+                (push (list new-state-start new-state-end state-symbol 
state-stack) new-states))
+            (if (> state-end start)
+                (let ((new-state-end (+ state-end diff)))
+                  (push (list state-start new-state-end state-symbol 
state-stack) new-states))
+              (push state-object new-states))))))
 
     new-states))
 
 (defun phps-mode-lexer-move-tokens (start diff)
   "Update tokens with moved lexer tokens after or equal to START with 
modification DIFF."
-  (setq phps-mode-lexer-tokens (phps-mode-lexer-get-moved-tokens 
phps-mode-lexer-tokens start diff)))
+  (when phps-mode-lexer-tokens
+    (setq phps-mode-lexer-tokens (phps-mode-lexer-get-moved-tokens 
phps-mode-lexer-tokens start diff))))
 
 (defun phps-mode-lexer-get-moved-tokens (old-tokens start diff)
   "Return moved lexer OLD-TOKENS positions after (or equal to) START with DIFF 
points."
@@ -1640,10 +1665,13 @@
               (let ((new-token-start (+ token-start diff))
                     (new-token-end (+ token-end diff)))
                 (push `(,token-symbol ,new-token-start . ,new-token-end) 
new-tokens))
-            (push token new-tokens)))))
-
+            (if (> token-end start)
+                (let ((new-token-end (+ token-end diff)))
+                  (push `(,token-symbol ,token-start . ,new-token-end) 
new-tokens))
+              (push token new-tokens))))))
     new-tokens))
 
+;; TODO Consider how imenu-index should be affected by this
 (defun phps-mode-lexer-run-incremental ()
   "Run incremental lexer based on 
`(phps-mode-functions-get-buffer-changes-start)'."
   ;; (message "Running incremental lexer")
@@ -1724,7 +1752,7 @@
   (when (boundp 'semantic-lex-analyzer)
     (setq semantic-lex-analyzer 'phps-mode-lexer-lex))
   (add-hook 'semantic-lex-reset-functions #'phps-mode-lexer-setup)
-  (set (make-local-variable 'phps-mode-lexer-tokens) nil)
+  (setq-local phps-mode-lexer-tokens nil)
   (phps-mode-lexer-run))
 
 (provide 'phps-mode-lexer)
diff --git a/phps-mode-semantic.el b/phps-mode-semantic.el
index 25a0eb5..0f29500 100644
--- a/phps-mode-semantic.el
+++ b/phps-mode-semantic.el
@@ -20,6 +20,7 @@
 
 ;;; Commentary:
 
+;; This entire file is WIP / TODO
 
 ;;; Code:
 
diff --git a/phps-mode-tags.el b/phps-mode-tags.el
index 9e13cf0..5ad3eb0 100644
--- a/phps-mode-tags.el
+++ b/phps-mode-tags.el
@@ -20,6 +20,8 @@
 
 ;;; Commentary:
 
+;; TODO This entire file is WIP / TODO
+
 ;; TODO Make similar to 
/Users/christianjohansson/Documents/emacs/nextstep/Emacs.app/Contents/Resources/lisp/cedet/semantic/wisent/java-tags.el.gz
 
 
@@ -29,61 +31,6 @@
 
 ;;;; Semantic integration of the PHP LALR parser
 
-;; In semantic/imenu.el, not part of Emacs.
-(defvar semantic-imenu-summary-function)
-
-;;;###autoload
-(defun phps-mode-tags-init ()
-  "Hook run to setup Semantic in `phps-mode'.
-Use the alternate LALR(1) parser."
-
-  (phps-mode-tags--install-parser)
-
-  (require 'semantic/bovine/debug)
-
-  (setq
-
-   ;; Semantic requires this expression for line-comments,
-   ;; if lexing without major mode
-   semantic-lex-comment-regex "\\s<\\|\\(/\\*\\|//\\)"
-
-   ;; Lexical analysis
-   semantic-lex-analyzer 'phps-mode-tags-lexer
-
-   ;; Parsing
-   semantic-tag-expand-function 'semantic-php-expand-tag
-
-   ;; Environment
-   semantic-imenu-summary-function 'semantic-format-tag-prototype
-   imenu-create-index-function 'semantic-create-imenu-index
-   semantic-type-relation-separator-character '("::" "->")
-   semantic-debug-parser-class 'semantic-bovine-debug-parser
-
-   ;; speedbar and imenu buckets name
-   semantic-symbol->name-assoc-list-for-type-parts
-
-   ;; in type parts
-   '((package . "Namespaces")
-     (type     . "Classes")
-     (variable . "Variables")
-     (function . "Functions"))
-   semantic-symbol->name-assoc-list
-
-   ;; everywhere
-   (append semantic-symbol->name-assoc-list-for-type-parts
-           '((namespace  . "Namespaces")))
-
-   ;; navigation inside 'type children
-   senator-step-at-tag-classes '(function variable)
-
-   ;; Remove 'recursive from the default semanticdb find throttle
-   ;; since php imports never recurse.
-   semanticdb-find-default-throttle
-
-   (remq 'recursive (default-value 'semanticdb-find-default-throttle)))
-
-  ;; Setup phpdoc stuff
-  (semantic-php-doc-setup))
 
 (provide 'phps-mode-tags)
 
diff --git a/phps-mode-test-functions.el b/phps-mode-test-functions.el
index 8a3b771..ded3512 100644
--- a/phps-mode-test-functions.el
+++ b/phps-mode-test-functions.el
@@ -25,16 +25,10 @@
 
 ;;; Code:
 
-(autoload 'phps-mode-test-with-buffer "phps-mode-test")
-(autoload 'phps-mode-functions-verbose "phps-mode-functions")
-(autoload 'phps-mode-functions-indent-line "phps-mode-functions")
-(autoload 'phps-mode-functions-get-lines-indent "phps-mode-functions")
-(autoload 'phps-mode-functions-get-imenu "phps-mode-functions")
-(autoload 'phps-mode-functions-get-moved-lines-indent "phps-mode-functions")
-(autoload 'phps-mode-test-hash-to-list "phps-mode-test")
-(autoload 'phps-mode-lexer-get-tokens "phps-mode-lexer")
-(autoload 'phps-mode-lexer-get-states "phps-mode-lexer")
-(autoload 'should "ert")
+(require 'ert)
+(require 'phps-mode-functions)
+(require 'phps-mode-lexer)
+(require 'phps-mode-test)
 
 (defun phps-mode-test-functions-move-lines-indent ()
   "Test `phps-mode-functions-move-lines-indent'."
@@ -247,6 +241,35 @@
    "Assignment with three-dimensional array with double arrow assignment"
    (should (equal '((1 (0 0)) (2 (0 0)) (3 (1 0)) (4 (2 0)) (5 (1 0)) (6 (0 
0))) (phps-mode-test-hash-to-list (phps-mode-functions-get-lines-indent)))))
 
+  (phps-mode-test-with-buffer
+   "<?php\nif ($myCondition) {\n    $myObject->myMethod(myClass::class)\n      
  ->myMethod2($myArgument2);\n    }"
+   "Object-oriented file with bracket-less namespace with multiple levels, 
class that extends and implements and functions with optional arguments"
+   (should (equal '((1 (0 0)) (2 (0 0)) (3 (1 0)) (4 (2 0)) (5 (0 0))) 
(phps-mode-test-hash-to-list (phps-mode-functions-get-lines-indent)))))
+
+  (phps-mode-test-with-buffer
+   "<?php\n$myObj->myFunction()\n    ->mySecondaryFunction();"
+   "Indentation of chained class method calls outside of assignments and 
conditionals"
+   ;; (message "Tokens: %s" (phps-mode-lexer-get-tokens))
+   (should (equal '((1 (0 0)) (2 (0 0)) (3 (1 0))) 
(phps-mode-test-hash-to-list (phps-mode-functions-get-lines-indent)))))
+
+  (phps-mode-test-with-buffer
+   "<?php\n\n$myVar = $myClass->meMethod()\n    ->mySecondMethod()\n    
->myThirdMethod()\n->myFourthFunction(\n    $myVariable\n);"
+   "Indentation for chained object operators in assignment with method call 
with arguments"
+   (should (equal '((1 (0 0)) (2 (0 0)) (3 (0 0)) (4 (1 0)) (5 (1 0)) (6 (1 
0)) (7 (2 0)) (8 (0 0))) (phps-mode-test-hash-to-list 
(phps-mode-functions-get-lines-indent)))))
+
+  (phps-mode-test-with-buffer
+   "<?php\n\n$myResult = !empty($myVar->myMethod3)\n    && $myVar->myMethod\n  
      && $myVar->myMethod2;\n"
+   "Indentation for chained object operators in assignment"
+   ;; (message "Tokens: %s" (phps-mode-lexer-get-tokens))
+   (should (equal '((1 (0 0)) (2 (0 0)) (3 (0 0)) (4 (1 0)) (5 (1 0))) 
(phps-mode-test-hash-to-list (phps-mode-functions-get-lines-indent)))))
+
+  (phps-mode-test-with-buffer
+   "<?php\n$array = [\n    'second' => [\n        'hello' => true\n        
]\n];\n\n$array = array(\n    'second' => array(\n        'third' => true\n     
   )\n);"
+   "Indent multi-dimensional arrays without trailing commas"
+   ;; (message "Tokens: %s" (phps-mode-lexer-get-tokens))
+   (should (equal '((1 (0 0)) (2 (0 0)) (3 (1 0)) (4 (2 0)) (5 (1 0)) (6 (0 
0)) (7 (0 0)) (8 (0 0)) (9 (1 0)) (10 (2 0)) (11 (1 0)) (12 (0 0))) 
(phps-mode-test-hash-to-list (phps-mode-functions-get-lines-indent))))
+   )
+
   )
 
 (defun phps-mode-test-functions-get-lines-indent-psr-2 ()
@@ -417,7 +440,7 @@
    "<?php\n$var =\n    500 .\n    \"200\" .\n    100.0 .\n    '200' .\n    
$this->getTail()\n    ->getBottom();"
    "Multi-line assignments"
    ;; (message "Tokens: %s" phps-mode-lexer-tokens)
-   (should (equal '((1 (0 0)) (2 (0 0)) (3 (1 0)) (4 (2 0)) (5 (2 0)) (6 (2 
0)) (7 (2 0)) (8 (2 0))) (phps-mode-test-hash-to-list 
(phps-mode-functions-get-lines-indent)))))
+   (should (equal '((1 (0 0)) (2 (0 0)) (3 (1 0)) (4 (2 0)) (5 (2 0)) (6 (2 
0)) (7 (2 0)) (8 (3 0))) (phps-mode-test-hash-to-list 
(phps-mode-functions-get-lines-indent)))))
 
   )
 
@@ -858,6 +881,34 @@
    "Imenu object-oriented file with bracket-less namespace with multiple 
levels, class that extends and implements and functions with optional arguments"
    (should (equal (phps-mode-functions-get-imenu) 
'(("myNamespace\\myNamespace2" ("myClass" ("myFunctionA" . 121) ("myFunctionB" 
. 174)))))))
 
+  (phps-mode-test-with-buffer
+   "<?php\nclass myClass\n{\n\n    public function myFunction1()\n    {\n      
  echo \"my string with variable {$variable} inside it\";\n    }\n\n    public 
function myFunction2()\n    {\n    }\n\n}"
+   "Imenu with double quoted string with variable inside it"
+   (should (equal (phps-mode-functions-get-imenu) '(("myClass" ("myFunction1" 
. 44)) ("myFunction2" . 153)))))
+
+  )
+
+(defun phps-mode-test-functions-get-moved-imenu ()
+  "Test for moving imenu index."
+
+  (message "Moved imenu %s" (phps-mode-functions-get-moved-imenu 
'(("myNamespace" ("myClass" ("myFunctionA" . 108) ("myFunctionB" . 161)))) 0 2))
+
+  (should (equal
+           '(("myNamespace" ("myClass" ("myFunctionA" . 110) ("myFunctionB" . 
163))))
+           (phps-mode-functions-get-moved-imenu '(("myNamespace" ("myClass" 
("myFunctionA" . 108) ("myFunctionB" . 161)))) 0 2)))
+
+  (should (equal
+           '(("myNamespace" ("myClass" ("myFunctionA" . 106) ("myFunctionB" . 
159))))
+           (phps-mode-functions-get-moved-imenu '(("myNamespace" ("myClass" 
("myFunctionA" . 108) ("myFunctionB" . 161)))) 0 -2)))
+
+  (should (equal
+           '(("myNamespace" ("myClass" ("myFunctionA" . 108) ("myFunctionB" . 
171))))
+           (phps-mode-functions-get-moved-imenu '(("myNamespace" ("myClass" 
("myFunctionA" . 108) ("myFunctionB" . 161)))) 110 10)))
+
+  (should (equal
+           '(("myNamespace" ("myClass" ("myFunctionA" . 108) ("myFunctionB" . 
161))))
+           (phps-mode-functions-get-moved-imenu '(("myNamespace" ("myClass" 
("myFunctionA" . 108) ("myFunctionB" . 161)))) 180 10)))
+
   )
 
 (defun phps-mode-test-functions-comment-uncomment-region ()
@@ -894,7 +945,7 @@
   )
 
 ;; TODO Add functionality for (delete-backward-char) as well
-;; TODO Add test for inserting newlines inside token
+;; TODO Test imenu movement here as well
 (defun phps-mode-test-functions-whitespace-modifications ()
   "Test white-space modifications functions."
 
@@ -954,6 +1005,31 @@
 
    )
 
+  (phps-mode-test-with-buffer
+   "<?php\nif (true):\n    $var = \"abc\nanother line here\nmore text 
here\";\n    $var2 = '123';\nendif;"
+   "Add test for inserting newlines inside token"
+
+   ;; (message "Before Tokens %s" (phps-mode-lexer-get-tokens))
+   ;; (message "Before States: %s" (phps-mode-lexer-get-states))
+
+   (should (equal (phps-mode-lexer-get-tokens)
+                  '((T_OPEN_TAG 1 . 7) (T_IF 7 . 9) ("(" 10 . 11) (T_STRING 11 
. 15) (")" 15 . 16) (":" 16 . 17) (T_VARIABLE 22 . 26) ("=" 27 . 28) 
(T_CONSTANT_ENCAPSED_STRING 29 . 67) (";" 67 . 68) (T_VARIABLE 73 . 78) ("=" 79 
. 80) (T_CONSTANT_ENCAPSED_STRING 81 . 86) (";" 86 . 87) (T_ENDIF 88 . 93) (";" 
93 . 94))))
+   (should (equal (phps-mode-lexer-get-states)
+                  '((93 94 1 (1 1 1 1 1)) (88 93 1 (1 1 1 1 1)) (86 87 1 (1 1 
1 1 1)) (81 86 1 (1 1 1 1 1)) (79 80 1 (1 1 1 1 1)) (73 78 1 (1 1 1 1 1)) (67 
68 1 (1 1 1 1 1)) (29 67 1 (1 1 1 1 1)) (27 28 1 (1 1 1 1 1)) (22 26 1 (1 1 1 1 
1)) (16 17 1 (1 1 1 1 1)) (15 16 1 (1 1 1 1 1)) (11 15 1 (1 1 1 1 1)) (10 11 1 
(1 1 1 1 1)) (7 9 1 (1 1 1 1 1)) (1 7 1 (1 1 1 1 1)))))
+
+   ;; Insert newline and then indent
+   (goto-char 51)
+   (newline-and-indent)
+
+   ;; (message "After Tokens %s" (phps-mode-lexer-get-tokens))
+   ;; (message "After States: %s" (phps-mode-lexer-get-states))
+   (should (equal (phps-mode-lexer-get-tokens)
+                  '((T_OPEN_TAG 1 . 7) (T_IF 7 . 9) ("(" 10 . 11) (T_STRING 11 
. 15) (")" 15 . 16) (":" 16 . 17) (T_VARIABLE 22 . 26) ("=" 27 . 28) 
(T_CONSTANT_ENCAPSED_STRING 29 . 76) (";" 76 . 77) (T_VARIABLE 82 . 87) ("=" 88 
. 89) (T_CONSTANT_ENCAPSED_STRING 90 . 95) (";" 95 . 96) (T_ENDIF 97 . 102) 
(";" 102 . 103))))
+   (should (equal (phps-mode-lexer-get-states)
+                  '((102 103 1 (1 1 1 1 1)) (97 102 1 (1 1 1 1 1)) (95 96 1 (1 
1 1 1 1)) (90 95 1 (1 1 1 1 1)) (88 89 1 (1 1 1 1 1)) (82 87 1 (1 1 1 1 1)) (76 
77 1 (1 1 1 1 1)) (29 76 1 (1 1 1 1 1)) (27 28 1 (1 1 1 1 1)) (22 26 1 (1 1 1 1 
1)) (16 17 1 (1 1 1 1 1)) (15 16 1 (1 1 1 1 1)) (11 15 1 (1 1 1 1 1)) (10 11 1 
(1 1 1 1 1)) (7 9 1 (1 1 1 1 1)) (1 7 1 (1 1 1 1 1)))))
+
+   )
+
   )
 
 (defun phps-mode-test-functions ()
@@ -970,6 +1046,7 @@
   (phps-mode-test-functions-get-lines-indent)
   (phps-mode-test-functions-indent-line)
   (phps-mode-test-functions-imenu)
+  (phps-mode-test-functions-get-moved-imenu)
   (phps-mode-test-functions-comment-uncomment-region)
   (phps-mode-test-functions-move-lines-indent)
   (phps-mode-test-functions-whitespace-modifications))
diff --git a/phps-mode-test-integration.el b/phps-mode-test-integration.el
index f8cbc16..93754a6 100644
--- a/phps-mode-test-integration.el
+++ b/phps-mode-test-integration.el
@@ -25,17 +25,11 @@
 
 ;;; Code:
 
-(autoload 'phps-mode-test-with-buffer "phps-mode-test")
-(autoload 'phps-mode-test-incremental-vs-intial-buffer "phps-mode-test")
-(autoload 'phps-mode-functions-get-lines-indent "phps-mode-functions")
-(autoload 'phps-mode-functions-get-imenu "phps-mode-functions")
-(autoload 'phps-mode-functions-get-buffer-changes-start "phps-mode-functions")
-(autoload 'phps-mode-lexer-get-tokens "phps-mode-lexer")
-(autoload 'phps-mode-lexer-run-incremental "phps-mode-lexer")
-(autoload 'phps-mode-test-hash-to-list "phps-mode-test")
-(autoload 'should "ert")
-
-;; TODO Add test for making changes inside tokens that is (and (> token-start) 
(< token-end))
+(require 'ert)
+(require 'phps-mode-functions)
+(require 'phps-mode-lexer)
+(require 'phps-mode-test)
+
 (defun phps-mode-test-integration-incremental ()
   "Test for object-oriented PHP file."
 
@@ -74,6 +68,22 @@
    (insert "<?php\nfunction myFunctionA()\n{\n    echo 'my second 
statement';\n}\n")
    (should (equal (phps-mode-functions-get-buffer-changes-start) 1)))
 
+  (phps-mode-test-incremental-vs-intial-buffer
+   "<?php\n/**\n *\n */\nnamespace myNamespace\n{\n    class myClass\n    {\n  
      public function myFunction()\n        {\n            echo 'my 
statement';\n        }\n    }\n}\n"
+   "Integration-test 4 for regular PHP with namespaces, classes and functions, 
white-space change inside token"
+
+   ;; (message "Before tokens: %s" (phps-mode-lexer-get-tokens))
+   ;; (message "Before indent: %s" (phps-mode-test-hash-to-list 
phps-mode-functions-lines-indent))
+
+   ;; Make changes
+   (goto-char 13)
+   (newline-and-indent)
+
+   ;; (message "After tokens: %s" (phps-mode-lexer-get-tokens))
+   ;; (message "After indent: %s" (phps-mode-test-hash-to-list 
phps-mode-functions-lines-indent))
+   
+   (should (equal (phps-mode-functions-get-buffer-changes-start) nil)))
+
   )
 
 (defun phps-mode-test-integration ()
diff --git a/phps-mode-test-lexer.el b/phps-mode-test-lexer.el
index 4d5faef..c4ed6af 100644
--- a/phps-mode-test-lexer.el
+++ b/phps-mode-test-lexer.el
@@ -25,13 +25,9 @@
 
 ;;; Code:
 
-(autoload 'phps-mode-test-with-buffer "phps-mode-test")
-(autoload 'phps-mode-lexer-init "phps-mode-lexer")
-(autoload 'phps-mode-lexer-get-point-data "phps-mode-lexer")
-(autoload 'phps-mode-lexer-get-moved-tokens "phps-mode-lexer")
-(autoload 'phps-mode-lexer-get-moved-states "phps-mode-lexer")
-(autoload 'phps-mode-lexer-get-tokens "phps-mode-lexer")
-(autoload 'should "ert")
+(require 'ert)
+(require 'phps-mode-lexer)
+(require 'phps-mode-test)
 
 (defun phps-mode-test-lexer-script-boundaries ()
   "Run test for lexer."
@@ -66,6 +62,13 @@
    (should (equal (phps-mode-lexer-get-tokens)
                   '((T_INLINE_HTML 1 . 39) (T_OPEN_TAG 39 . 45) (T_EXIT 47 . 
51) (";" 51 . 53) (T_CLOSE_TAG 51 . 53) (T_INLINE_HTML 53 . 78) (T_OPEN_TAG 78 
. 84) (T_EXIT 84 . 88) (";" 89 . 91) (T_CLOSE_TAG 89 . 91)))))
 
+  (phps-mode-test-with-buffer
+   "<?php\n\n$k = 'key';\n\necho \"\\$a['{$k}']\";"
+   "A tricky case where variable inside double quote is escaped"
+   ;; (message "Tokens: %s" (phps-mode-lexer-get-tokens))
+   (should (equal (phps-mode-lexer-get-tokens)
+                  '((T_OPEN_TAG 1 . 7) (T_VARIABLE 8 . 10) ("=" 11 . 12) 
(T_CONSTANT_ENCAPSED_STRING 13 . 18) (";" 18 . 19) (T_ECHO 21 . 25) ("\"" 26 . 
27) (T_ENCAPSED_AND_WHITESPACE 27 . 32) (T_CURLY_OPEN 32 . 33) (T_VARIABLE 33 . 
35) ("}" 35 . 36) (T_CONSTANT_ENCAPSED_STRING 36 . 38) ("\"" 38 . 39) (";" 39 . 
40)))))
+
   )
 
 (defun phps-mode-test-lexer-simple-tokens ()
@@ -346,24 +349,24 @@
   "Run test for get moved lexer tokens."
 
   (should (equal
-           '((T_OPEN_TAG 1 . 7) (T_START_HEREDOC 7 . 16) (T_ERROR 21 . 60))
+           '((T_OPEN_TAG 1 . 7) (T_START_HEREDOC 7 . 21) (T_ERROR 21 . 60))
            (phps-mode-lexer-get-moved-tokens '((T_OPEN_TAG 1 . 7) 
(T_START_HEREDOC 7 . 16) (T_ERROR 16 . 55)) 8 5)))
 
   (should (equal
-           '((T_OPEN_TAG 1 . 7) (T_START_HEREDOC 7 . 16) (T_ERROR 11 . 50))
+           '((T_OPEN_TAG 1 . 7) (T_START_HEREDOC 7 . 11) (T_ERROR 11 . 50))
            (phps-mode-lexer-get-moved-tokens '((T_OPEN_TAG 1 . 7) 
(T_START_HEREDOC 7 . 16) (T_ERROR 16 . 55)) 8 -5)))
 
   (should (equal
-           '((T_OPEN_TAG 1 . 7) (T_START_HEREDOC 8 . 17) (T_ERROR 17 . 56))
-           (phps-mode-lexer-get-moved-tokens '((T_OPEN_TAG 1 . 7) 
(T_START_HEREDOC 7 . 16) (T_ERROR 16 . 55)) 7 1)))
+           '((T_OPEN_TAG 1 . 8) (T_START_HEREDOC 8 . 17) (T_ERROR 17 . 56))
+           (phps-mode-lexer-get-moved-tokens '((T_OPEN_TAG 1 . 7) 
(T_START_HEREDOC 7 . 16) (T_ERROR 16 . 55)) 6 1)))
 
   (should (equal
-           '((T_OPEN_TAG 1 . 7) (T_START_HEREDOC 7 . 16) (T_ERROR 16 . 55))
+           '((T_OPEN_TAG 1 . 7) (T_START_HEREDOC 7 . 16) (T_ERROR 16 . 56))
            (phps-mode-lexer-get-moved-tokens '((T_OPEN_TAG 1 . 7) 
(T_START_HEREDOC 7 . 16) (T_ERROR 16 . 55)) 20 1)))
 
   (should (equal
-           '((T_OPEN_TAG 2 . 8) (T_START_HEREDOC 8 . 17) (T_ERROR 17 . 56))
-           (phps-mode-lexer-get-moved-tokens '((T_OPEN_TAG 1 . 7) 
(T_START_HEREDOC 7 . 16) (T_ERROR 16 . 55)) -20 1)))
+           '((T_OPEN_TAG 1 . 7) (T_START_HEREDOC 7 . 16) (T_ERROR 16 . 54))
+           (phps-mode-lexer-get-moved-tokens '((T_OPEN_TAG 1 . 7) 
(T_START_HEREDOC 7 . 16) (T_ERROR 16 . 55)) 20 -1)))
 
   )
 
@@ -373,13 +376,13 @@
   (should (equal
            '((68 76 1 '(1))
              (10 67 1 '(1))
-             (1 7 1 '(1)))
+             (1 9 1 '(1)))
            
            (phps-mode-lexer-get-moved-states
             '((66 74 1 '(1))
               (8 65 1 '(1))
               (1 7 1 '(1)))
-            7
+            6
             2)))
 
   (should (equal
diff --git a/phps-mode-test-syntax-table.el b/phps-mode-test-syntax-table.el
index 76a8207..b4b81db 100644
--- a/phps-mode-test-syntax-table.el
+++ b/phps-mode-test-syntax-table.el
@@ -25,13 +25,9 @@
 
 ;;; Code:
 
-(autoload 'phps-mode-test-with-buffer "phps-mode-test")
-(autoload 'phps-mode-functions-verbose "phps-mode-functions")
-(autoload 'phps-mode-functions-indent-line "phps-mode-functions")
-(autoload 'phps-mode-functions-get-lines-indent "phps-mode-functions")
-(autoload 'phps-mode-functions-get-imenu "phps-mode-functions")
-(autoload 'phps-mode-test-hash-to-list "phps-mode-test")
-(autoload 'should "ert")
+(require 'ert)
+(require 'phps-mode-functions)
+(require 'phps-mode-test)
 
 (defun phps-mode-test-syntax-table-quote-region ()
   "Test double quotes, single quotes, curly bracket, square bracket, round 
bracket, back-quotes on regions."
diff --git a/phps-mode-test.el b/phps-mode-test.el
index 3289877..e9098f4 100644
--- a/phps-mode-test.el
+++ b/phps-mode-test.el
@@ -23,26 +23,25 @@
 
 ;;; Code:
 
-(autoload 'phps-mode "phps-mode")
-(autoload 'phps-mode-lexer-get-tokens "phps-mode-lexer")
-(autoload 'phps-mode-functions-get-imenu "phps-mode-functions")
-(autoload 'phps-mode-functions-get-lines-indent "phps-mode-functions")
-(autoload 'should "ert")
+(require 'ert)
+(require 'phps-mode)
+(require 'phps-mode-functions)
+(require 'phps-mode-lexer)
 
 (defmacro phps-mode-test-incremental-vs-intial-buffer (source &optional title 
&rest change)
   "Set up test buffer with SOURCE, TITLE, apply CHANGE and compare incremental 
values with initial values."
   `(let ((test-buffer-incremental (generate-new-buffer "test-incremental"))
+         (incremental-states nil)
          (incremental-tokens nil)
          (incremental-imenu nil)
          (incremental-indent nil)
          (incremental-buffer nil)
-         (incremental-overlays nil)
          (test-buffer-initial (generate-new-buffer "test-initial"))
+         (initial-states nil)
          (initial-tokens nil)
          (initial-imenu nil)
          (initial-indent nil)
-         (initial-buffer nil)
-         (initial-overlays nil))
+         (initial-buffer nil))
 
      ;; Setup incremental buffer
      (switch-to-buffer test-buffer-incremental)
@@ -54,12 +53,11 @@
      (phps-mode)
      ,@change
      (phps-mode-lexer-run-incremental)
+     (setq incremental-states (phps-mode-lexer-get-states))
      (setq incremental-tokens (phps-mode-lexer-get-tokens))
      (setq incremental-imenu (phps-mode-functions-get-imenu))
      (setq incremental-indent (phps-mode-test-hash-to-list 
(phps-mode-functions-get-lines-indent)))
      (setq incremental-buffer (buffer-substring-no-properties (point-min) 
(point-max)))
-     (setq incremental-overlays (overlays-in (point-min) (point-max)))
-     (kill-buffer test-buffer-incremental)
 
      ;; Setup incremental buffer
      (switch-to-buffer test-buffer-initial)
@@ -69,24 +67,29 @@
                 phps-mode-functions-verbose)
        (message "\nTesting initial buffer '%s':\n'%s'\n" ,title 
incremental-buffer))
      (phps-mode)
+     (setq initial-states (phps-mode-lexer-get-states))
      (setq initial-tokens (phps-mode-lexer-get-tokens))
      (setq initial-imenu (phps-mode-functions-get-imenu))
      (setq initial-indent (phps-mode-test-hash-to-list 
(phps-mode-functions-get-lines-indent)))
      (setq initial-buffer (buffer-substring-no-properties (point-min) 
(point-max)))
-     (setq initial-overlays (overlays-in (point-min) (point-max)))
-     (kill-buffer test-buffer-initial)
 
      ;; Run tests
      (when (and (boundp 'phps-mode-functions-verbose)
                 phps-mode-functions-verbose)
-       (message "\nComparing tokens, lines indent, imenu and overlays between 
buffer:\n\n'%s'\n\nand:\n\n'%s'\n" initial-buffer incremental-buffer))
+       (message "\nComparing tokens, lines indent and imenu  between 
buffer:\n\n'%s'\n\nand:\n\n'%s'\n" initial-buffer incremental-buffer))
      (should (equal initial-buffer incremental-buffer))
      ;; (message "Initial tokens: %s\n" initial-tokens)
      ;; (message "Incremental tokens: %s\n" incremental-tokens)
+     (should (equal initial-states incremental-states))
      (should (equal initial-tokens incremental-tokens))
+     ;; (message "Initial indent: %s\n" initial-indent)
+     ;; (message "Incremental indent: %s\n" incremental-indent)
      (should (equal initial-indent incremental-indent))
      (should (equal initial-imenu incremental-imenu))
-     (should (equal initial-overlays incremental-overlays))
+
+
+     (kill-buffer test-buffer-incremental)
+     (kill-buffer test-buffer-initial)
 
      (when ,title
        (message "\nPassed incremental tests for '%s'\n" ,title))))
diff --git a/phps-mode.el b/phps-mode.el
index fa6df56..29afbd4 100644
--- a/phps-mode.el
+++ b/phps-mode.el
@@ -5,8 +5,8 @@
 ;; Author: Christian Johansson <github.com/cjohansson>
 ;; Maintainer: Christian Johansson <github.com/cjohansson>
 ;; Created: 3 Mar 2018
-;; Modified: 17 Jul 2019
-;; Version: 0.2.3
+;; Modified: 16 Aug 2019
+;; Version: 0.2.4
 ;; Keywords: tools, convenience
 ;; URL: https://github.com/cjohansson/emacs-phps-mode
 
@@ -25,7 +25,7 @@
 ;; General Public License for more details.
 
 ;; You should have received a copy of the GNU General Public License
-;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
+;; along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.
 
 
 ;;; Commentary:
@@ -40,52 +40,110 @@
 
 ;; NOTE use wisent-parse-toggle-verbose-flag and (semantic-debug) to debug 
parsing
 
-(autoload 'phps-mode-flycheck-init "phps-mode-flycheck")
-(autoload 'phps-mode-flymake-init "phps-mode-flymake")
-(autoload 'phps-mode-font-lock-init "phps-mode-font-lock")
-(autoload 'phps-mode-functions-init "phps-mode-functions")
-(autoload 'phps-mode-map-init "phps-mode-map")
-(autoload 'phps-mode-lexer-init "phps-mode-lexer")
-(autoload 'phps-mode-syntax-table-init "phps-mode-syntax-table")
-(autoload 'phps-mode-tags-init "phps-mode-tags")
-(autoload 'phps-mode-semantic-init "phps-mode-semantic")
-
-(autoload 'semantic-new-buffer-fcn "semantic")
+(require 'phps-mode-flymake)
+(require 'phps-mode-functions)
+(require 'phps-mode-lexer)
+(require 'phps-mode-semantic)
+(require 'phps-mode-syntax-table)
+(require 'phps-mode-tags)
+(require 'semantic)
 
 (defvar phps-mode-use-psr-2 t
   "Whether to use PSR-2 guidelines for white-space or not.")
 
 (defvar phps-mode-idle-interval 1.0
-  "Idle seconds before running incremental lexer.")
+  "Idle seconds before running the incremental lexer.")
+
+(defvar phps-mode-flycheck-applied nil "Boolean flag whether flycheck 
configuration has been applied or not.")
+
+(defvar phps-mode-map-applied nil "Boolean flag whether mode-map has been 
initialized or not.")
+
+(defvar phps-mode-inline-mmm-submode nil "Symbol declaring what mmm-mode to 
use as submode in inline areas.")
 
 (define-derived-mode phps-mode prog-mode "PHPs"
   "Major mode for PHP with Semantic integration."
 
+  ;; TODO Check whether PSR-2 requires final newlines or not
+  (setq-local require-final-newline nil)
+
+  ;; TODO Verify this setting
+  (setq-local parse-sexp-ignore-comments nil)
+
   ;; Key-map
-  (phps-mode-map-init)
+  ;; prog-mode will create the key-map and we just modify it here.
+  (when (and phps-mode-map
+             (not phps-mode-map-applied))
+    (define-key phps-mode-map (kbd "C-c /") #'comment-region)
+    (define-key phps-mode-map (kbd "C-c DEL") #'uncomment-region)
+    (setq phps-mode-map-applied t))
+  (use-local-map phps-mode-map)
 
   ;; Syntax table
   (phps-mode-syntax-table-init)
 
   ;; Font lock
-  (phps-mode-font-lock-init)
+  ;; This makes it possible to have full control over syntax coloring from the 
lexer
+  (setq-local font-lock-keywords-only nil)
+  (setq-local font-lock-defaults '(nil t))
 
   ;; Flymake TODO
   ;; (phps-mode-flymake-init)
 
   ;; Flycheck
-  (phps-mode-flycheck-init)
-
-  ;; Override functions
-  (phps-mode-functions-init)
+  ;; Add support for flycheck PHP checkers: PHP, PHPMD and PHPCS here, do it 
once but only if flycheck is available
+  (when (and (fboundp 'flycheck-add-mode)
+             (not phps-mode-flycheck-applied))
+    (flycheck-add-mode 'php 'phps-mode)
+    (flycheck-add-mode 'php-phpmd 'phps-mode)
+    (flycheck-add-mode 'php-phpcs 'phps-mode)
+    (setq phps-mode-flycheck-applied t))
+
+    ;; Custom indentation
+  ;; NOTE Indent-region will call this on each line of region
+  (setq-local indent-line-function #'phps-mode-functions-indent-line)
+
+  ;; Custom Imenu
+  (setq-local imenu-create-index-function 
#'phps-mode-functions-imenu-create-index)
+
+  ;; Should we follow PSR-2?
+  (when (and (boundp 'phps-mode-use-psr-2)
+             phps-mode-use-psr-2)
+
+    ;; Code MUST use an indent of 4 spaces
+    (setq-local tab-width 4)
+
+    ;; MUST NOT use tabs for indenting
+    (setq-local indent-tabs-mode nil))
+
+  ;; Add support for moving indexes quickly when making newlines
+  (advice-add #'newline :around #'phps-mode-functions-around-newline)
+
+  ;; Reset flags
+  (set (make-local-variable 'phps-mode-functions-allow-after-change) t)
+  (set (make-local-variable 'phps-mode-functions-buffer-changes-start) nil)
+  (set (make-local-variable 'phps-mode-functions-lines-indent) nil)
+  (set (make-local-variable 'phps-mode-functions-imenu) nil)
+  (set (make-local-variable 'phps-mode-functions-processed-buffer) nil)
+
+  ;; Make (comment-region) and (uncomment-region) work
+  (setq-local comment-region-function #'phps-mode-functions-comment-region)
+  (setq-local uncomment-region-function #'phps-mode-functions-uncomment-region)
+  (setq-local comment-start "// ")
+  (setq-local comment-end "")
+
+  ;; Support for change detection
+  (add-hook 'after-change-functions #'phps-mode-functions-after-change)
 
   ;; Lexer
   (phps-mode-lexer-init)
 
-  ;; Wisent LALR parser
+  ;; Wisent LALR parser TODO
   ;; (phps-mode-tags-init)
 
+  ;; Add compatibility for plug-ins here
   (run-hooks 'phps-mode-hook)
+
+  ;; Run semantic functions for new buffer
   (semantic-new-buffer-fcn))
 
 (provide 'phps-mode)
diff --git a/sample-php-files/alternative-control-structure.php 
b/sample-php-files/alternative-control-structure.php
index 817c61f..cd9a48a 100644
--- a/sample-php-files/alternative-control-structure.php
+++ b/sample-php-files/alternative-control-structure.php
@@ -21,3 +21,8 @@ switch (true):
         echo 'the default';
 endswitch;
 
+?>
+<html>
+    <body>
+    </body>
+</html>
diff --git a/sample-php-files/functions.php b/sample-php-files/functions.php
index 3cf263f..76199ea 100644
--- a/sample-php-files/functions.php
+++ b/sample-php-files/functions.php
@@ -3,6 +3,9 @@
 /**
  * @param string $myArg1
  * @param string $myArg2
+ 
+
+ 
  */
 function myFunctionA($myArg1, $myArg2)
 {
@@ -10,3 +13,17 @@ function myFunctionA($myArg1, $myArg2)
     $myArg2 = false;
     echo "some stuff";
 }
+
+function myFunctionB($myArg3) {
+    echo 'something';
+}
+
+?>
+<body>
+    <div>
+        <strong>Hello</strong>
+        <p>
+            More stuff
+        </p>
+    </div>
+</body>
\ No newline at end of file



reply via email to

[Prev in Thread] Current Thread [Next in Thread]