swiper: (minibuffer-keyboard-quit) jumps back to first swiper match instead of point when calling isearch

[fapdash]

Reproduction

Config:

(use-package ace-isearch
  :ensure t
  :config
  (global-ace-isearch-mode +1)

  (custom-set-variables
   '(ace-isearch-input-length 2)
   '(ace-isearch-jump-delay 0.15)
   '(ace-isearch-function 'avy-goto-word-1)
   '(ace-isearch-use-jump 'printing-char)
   '(ace-isearch-function-from-isearch 'ace-isearch-swiper-from-isearch)))

• Execute isearch-forward with more than 1 char that has at least one match in the file so we change position
• Call minibuffer-keyboard-quit (C-g)

Expected

We jump back to the position where we called isearch-forward

Actual

We jump back to the first match from swiper where we jumped to after swiper was called..

[fapdash]

I fixed this in my own config through

(defun ace-isearch/jump-back-to-point ()
    (setq ace-isearch--opoint (point))

(add-hook 'isearch-mode-hook 'ace-isearch/jump-back-to-point)
(add-hook 'isearch-mode-end-hook (lambda () (unless isearch-mode-end-hook-quit (goto-char ace-isearch--opoint)))

What confuses me is that when I've read the docs correctly then I should be using when instead of unless as isearch-mode-end-hook-quit should be non-nil in the case of quitting but that's how I had to write it.