Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Epic: Build universal app to support Apple Silicon #1136

Closed
5 tasks done
ychin opened this issue Dec 10, 2020 · 6 comments
Closed
5 tasks done

Epic: Build universal app to support Apple Silicon #1136

ychin opened this issue Dec 10, 2020 · 6 comments
Labels
Epic Large tasks to be split up into smaller chunks Infrastructure Non-app infrastructure issues, e.g. CI

Comments

@ychin
Copy link
Member

ychin commented Dec 10, 2020

We need to get an universal app built for MacVim that will work both on Apple Silicon / M1 Macs. Currently MacVim is buildable under Apple Silicon, but the binary release is only x86 and will work in Apple Silicon under Rosetta. This is usually not a big deal but it's less efficient, and it makes it impossible to interface with native Python / Ruby / etc in plugins.

Tracking some tasks that we need to do or dependencies:

  • Set up CI support for Apple Silicon #1263
  • Homebrew support for Apple Silicon. We mostly need it to work for external packages like gettext and other scripting languages.
    • They already support Apple Silicon in unofficial manner, with /opt/homebrew/ currently the tentative folder for arm64 packages (since Homebrew is planning to only support thin binaries for now, they have to put these packages at a difference place).
  • Support for building universal x86 / Apple Silicon (arm64) app #1150
    • Modify CI build scripts to generate universal app. MacVim itself is easy via a single Xcode setting, but to build Vim itself we need to either get the compiler to generate universal binary or use lipo to stitch it together. Also, it's unclear if each CI instance builds their own non-universal app and we combine them in a later CI step, or each CI builds an universal app separately (which is annoying because then we need to cross-compile with all the dependencies available).
  • Update Homebrew-core's formula, so brew install macvim also gets Apple Silicon version.
  • Documentation for users for how to use this, turn on/off, configure scripting languages.
    • Thinking more about this, I don't know what this actually means. MacVim works right now and there really isn't much you need to know about as a user. Just crossing this off.
@ychin ychin added this to the snapshot-168 milestone Dec 10, 2020
@ichizok
Copy link
Member

ichizok commented Dec 10, 2020

For building Vim as universal:

  • set CC=xcrun --sdk macosx11.0 clang (on 10.15. on 11.0 is ok to default)
  • append -arch arm64e -arch x86_64 to CFLAGS and LDFLAGS

ref:
https://github.com/ichizok/macvim/tree/build-universal
https://travis-ci.com/github/ichizok/macvim/jobs/445399373

@ychin
Copy link
Member Author

ychin commented Dec 10, 2020

@ichizok Yes it will work, but as I mentioned, we still need to get both ARM and x86 versions of dependencies (gettext, python, etc) on the same build machine since Vim needs to be built against them. Right now we use Homebrew to do it, but I don't think it quite has full support for doing that yet. We also need to make sure the Apple Silicon version of MacVim knows where to grab pythonthreedll from correctly.

ychin added a commit that referenced this issue Dec 11, 2020
Updated to Vim 8.2.2127.

*Note*: This release doesn't natively support Apple Silicon / M1 yet, but
does work under Rosetta. See below.

Features
====================

Big Sur / macOS 11
--------------------

- MacVim now has an updated app icon (#1054), and preference pane / toolbars
  have been updated to match Big Sur's interface guidelines. (#1128)
- Fixed Touch Bar warnings when launching MacVim from the terminal.
  #1114
- SF Symbol characters will show up properly as double-width as most of
  these icons would take up more than one column. Note that these
  characters are specific to macOS and would not work in other
  platforms. #1129

Renderer / scrolling performance improvements
--------------------

The Core Text renderer has been rewritten and is now much faster!
Scrolling should not stutter and lag like before and generally it should
feel a lot smoother now. Thanks to Sidney San Martín (@s4y) for the
contribution. #858

With this change, the non-Core-Text renderer is now considered
deprecated. The old renderer is accessible either through the Preference
Pane (under Advanced) or by setting the defaults "MMRenderer" to 0. It
works for now, but it will be removed in a future update as it has known
bugs.

Menu Localization
--------------------

Menus are now localized, see `:h langmenu` for how Vim menu localization
works. You can use `set langmenu=none` to turn it off if you would like. #1099

There still exists a few menu items that are not localized, and the
general MacVim GUI is not localized as well. If you would like to help,
please use #1102 to coordinate with MacVim dev team.

Getting help / Help menu
--------------------

- Help menu's search bar now searches Vim documentation as well! See
  #1095.
- Vimtutor is now bundled with MacVim, and you can access vimtutor from
  the Help menu (#1096). There is also a link to the latest release
  notes as well (#1131).

General
====================

- This release does not contain a native universal app for Apple Silicon
  / M1 Macs yet. The release binary will still work under Rosetta, which
  should provide enough performance, but if you use Python/etc plugins,
  you need to make sure you have x86 versions of Python/etc installed
  (which is still the default for Homebrew as of this release).

  MacVim is buildable under Apple Silicon, so if you need a native
  binary, you could build it yourself by downloading the source from the
  Github repository. See #1136 for progress on releasing a universal app
  for Apple Silicon.

- MacVim has enabled the Github Discussions feature, which serves as a
  good spot for general discussions and questions. See
  #1130 and check it
  out!

Fixes
====================

- Launching MacVim from the Dock with locales that use "," for decimal
  separators now works correctly. #11 (Vim 8.2.1738)
- `WinBar` menus (which are used by plugins like vimspector) now work
  properly and don't create dummy menu items. #918
- Using `:browse tabnew` no longer crashes MacVim in terminal mode.
  #1107 (Vim 8.2.1842)

Misc
====================

- Scripting languages versions:
    - Python is now built against 3.9, up from 3.8.
    - Lua is now built against 5.4, up from 5.3.

Compatibility
====================

Requires macOS 10.9 or above.

Script interfaces have compatibility with these versions:

- Lua 5.4
- Perl 5.18
- Python2 2.7
- Python3 3.9
- Ruby 2.7
@ychin
Copy link
Member Author

ychin commented Dec 12, 2020

Actually I'm not sure if we are supposed to use arm64e. I remember when I looked into it a few months ago, it's not really an officially supported target yet, and Apple's docs just calls for arm64.

Also, we don't need to add to CFLAGS I think. configure script already supports a --with-macarchs flag to do that for us.

ychin added a commit that referenced this issue Dec 12, 2020
Updated to Vim 8.2.2127.

*Note*: This release doesn't natively support Apple Silicon / M1 yet, but
does work under Rosetta. See below.

Features
====================

Big Sur / macOS 11
--------------------

- MacVim now has an updated app icon (#1054), and preference pane / toolbars
  have been updated to match Big Sur's interface guidelines. (#1128)
- Fixed Touch Bar warnings when launching MacVim from the terminal.
  #1114
- SF Symbol characters will show up properly as double-width as most of
  these icons would take up more than one column. Note that these
  characters are specific to macOS and would not work in other
  platforms. #1129

Renderer / scrolling performance improvements
--------------------

The Core Text renderer has been rewritten and is now much faster!
Scrolling should not stutter and lag like before and generally it should
feel a lot smoother now. Thanks to Sidney San Martín (@s4y) for the
contribution. #858

With this change, the non-Core-Text renderer is now considered
deprecated. The old renderer is accessible either through the Preference
Pane (under Advanced) or by setting the defaults "MMRenderer" to 0. It
works for now, but it will be removed in a future update as it has known
bugs.

Menu Localization
--------------------

Menus are now localized, see `:h langmenu` for how Vim menu localization
works. You can use `set langmenu=none` to turn it off if you would like. #1099

There still exists a few menu items that are not localized, and the
general MacVim GUI is not localized as well. If you would like to help,
please use #1102 to coordinate with MacVim dev team.

Getting help / Help menu
--------------------

- Help menu's search bar now searches Vim documentation as well! See
  #1095.
- Vimtutor is now bundled with MacVim, and you can access vimtutor from
  the Help menu (#1096). There is also a link to the latest release
  notes as well (#1131).

General
====================

- This release does not contain a native universal app for Apple Silicon
  / M1 Macs yet. The release binary will still work under Rosetta, which
  should provide enough performance, but if you use Python/etc plugins,
  you need to make sure you have x86 versions of Python/etc installed
  (which is still the default for Homebrew as of this release).

  MacVim is buildable under Apple Silicon, so if you need a native
  binary, you could build it yourself by downloading the source from the
  Github repository. See #1136 for progress on releasing a universal app
  for Apple Silicon.

- MacVim has enabled the Github Discussions feature, which serves as a
  good spot for general discussions and questions. See
  #1130 and check it
  out!

Fixes
====================

- Launching MacVim from the Dock with locales that use "," for decimal
  separators now works correctly. #11 (Vim 8.2.1738)
- `WinBar` menus (which are used by plugins like vimspector) now work
  properly and don't create dummy menu items. #918
- Using `:browse tabnew` no longer crashes MacVim in terminal mode.
  #1107 (Vim 8.2.1842)

Misc
====================

- Scripting languages versions:
    - Python is now built against 3.9, up from 3.8.
    - Lua is now built against 5.4, up from 5.3.

Compatibility
====================

Requires macOS 10.9 or above.

Script interfaces have compatibility with these versions:

- Lua 5.4
- Perl 5.18
- Python2 2.7
- Python3 3.9
- Ruby 2.7
@ichizok
Copy link
Member

ichizok commented Dec 14, 2020

I didn't know --with-macarchs... thank you.
As to arm64e, since Apple native apps and executables are built for arm64e (and x86_64) so I just thought of using it and not particular about it.
And I overlooked Sparkle is built for arm64 but not arm64e thus MacVim cannot be for arm64e now.
(Speaking of this point, using CFLAGS is applied to MacVim.app/Contents/MacOS/Vim and not to MacVim so then Vim is built for arm64e/x86_64 and MacVim is for arm64/x86_64)

@ychin ychin added the Infrastructure Non-app infrastructure issues, e.g. CI label Dec 19, 2020
ychin added a commit to ychin/macvim that referenced this issue Dec 22, 2020
This adds support for building MacVim as a fat binary (aka universal
app) for x86_64 / arm64 in CI.

The main challenge mostly lies in configuring the scripting language
default search paths for the libraries, and linking against gettext.
There are two possible approaches:
1. configure/build each arch completely separately, and then use `lipo`
   to stitch them back together. This is pretty annoying to set up, and
   kind of manual to do, and requires building the same thing twice,
   which is not great.
2. Build once with `--with-macarchs="x86_64 arm64` flag, which is what
   we do here.

gettext: Homebrew doesn't support fat binaries, and we also need to
build a custom x86 version of gettext to support down to macOS 10.9
anyway, so we manually download the bottle for arm64 gettext bottle, and
then stitch it with the x86 version to create a unified binary under
/usr/local/lib. This way we can just link against it in one go.

Scripting languages: Add new ifdef's to load different libs under
different architecture. Modify configure to support that (instead of
hacking a patch in during CI like Ruby). This means while on x86_64 it
will look under /usr/local/lib for Python 3, on arm64 it will look under
/opt/homebrew instead (this is the recommended path for Homebrew
installs for native arm64 packages). This new path is very specific to
Homebrew which is not ideal, but we could change this later and maybe
make the default search path logic for scripting languages smarter.

Note that since there is no arm64 in CI right now, this just builds the
app, but there will be no automatic testing to make sure it actually
works.

This is part of macvim-dev#1136.
@ychin
Copy link
Member Author

ychin commented Dec 23, 2020

We are currently building for Apple Silicon in CI! It will build the app, but no testing support though since we don't have M1 Macs in CI yet. Testing will have to be done manually.

@ychin ychin changed the title Build universal app to support Apple Silicon Epic: Build universal app to support Apple Silicon Dec 23, 2020
@ychin ychin modified the milestones: snapshot-170, snapshot-171 Mar 8, 2021
@ychin ychin modified the milestones: snapshot-171, snapshot-172 Apr 1, 2021
@ychin ychin modified the milestones: snapshot-174, Backlog Aug 5, 2022
@ychin ychin closed this as completed in c53e790 Jan 2, 2023
@ychin
Copy link
Member Author

ychin commented Mar 23, 2023

Ok this is bad. Merging upstream Vim changes can cause GitHub issue closure markers to trigger, which closed this issue from a completely unrelated Vim issue.

Edit: See https://github.com/orgs/community/discussions/17308

@ychin ychin reopened this Mar 23, 2023
@ychin ychin added the Epic Large tasks to be split up into smaller chunks label Oct 25, 2023
@ychin ychin closed this as completed Jan 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Epic Large tasks to be split up into smaller chunks Infrastructure Non-app infrastructure issues, e.g. CI
Projects
None yet
Development

No branches or pull requests

2 participants