After a long struggle with this problem, and observing no conclusive solution in many posts on both Ableton and Cycling74 forums, this is probably the beginning of another debate, but here at least is a solution that apparently works.

If you're putting externals in a project, you're going to see red, no matter what you do
If you're putting externals in a project, you're going to see red, no matter what you do

...If you are reading this, you probably already know the problem, but to summarize: many standalone and M4l applications use either externally developed binaries, or need access to files not in the patcher. These latter files may include poly~ subpatchers, coll data, pattrstorage data, audio media, video media, and other forms of 'local' data which is not stored inside the Max patcher itself. From one perspective, whether the file is another subpatcher, or a binary, or text data, or audio data, or a movie, should essentially make no difference in theory, as they are all 'external' files per se. In practice, as a build project needs to manage many other files besides the user's, the build process sorts external files for standalones, collectives, and M4L .amxd files, and handles each type slightly differenty.

The most common way to add one's own externals is by including them in a project. By including an external in the project file list, the build process should be able to add it to the target. However, to do so, it must put it somewhere, and so the build process creates subdirectories for subpatchers, media and data, then attempts to move the files there. Unfortunately, the original design may have put the files in a different place (mostly, in the same directory, rather than another one, except for external binaries, which are usually in a special search path location). So the build process, when it creates a frozen file or standalone, munges links and references to the externals in an attempt to keep the result behaving the same way as the original. (That is, I think it is called 'munging', but opinions may differ on that). And it does so transparently, because, for example, M4L users don't want to know that there are ghost copies of their work floating around in different directories eleswhere. They just want to unfreeze, edit, and save.

M4L Project Behavior

In Max for Live 6.1.9, when unfreezing a device, quite a bit happens under the hood. Most importantly, a ghost project file is generated invisibly and automatically (and not stored by default). One can view and save the project file by pressing a button with folder picture on it, in the toolbar of the top-le3velpatcher for the unfrozen .amxd device. The project settings need to be saved somehow, if they are changed manually, or re-entered very time a patch is opened for edit.

A 6.1.9 M4L project, by default, stores data files to a data subdirectory; media files to a media subdirectory; and subpatches (which apparently are assumed to be bpatchers), to a 'patchers' subdirectory. It's not expected that the device is polyphonic, so you will probably see some some red errors in the console, but they are not pertinent for poly~. If the bpatcher is embedded in the parent, rather than linked externally, the project may also display errors in the Max console (or the Max for Live console, both of which you need to keep open if you are building a project and see all errors). If a subpatcher has a UI, the project will assume it is meant to be in the top-level M4L patch, and will again display an error in the console.

When Max 6 was first released, settings for locations of externals was set in the search path, but in 6.1+ this behavior was deprecated, so many of the older forum posts on this topic are not pertinent any more. Now it is all managed by the project file, and it appears the default creation of these subdirectories for subpatchers, media, and data is no longer under user control, although you may be able to override the default file shuffling by changing a setting in the project 'attempt to reorganize files,' the action of which is controlled by a popup panel from the toolbar menu under the project's 'info' button. Unfortunately, though, if you then rename the .amxd file, any project file you made will no longer work for the renamed file, because it is hardlinked to the original filename, and so then you will have to create the project file again.

The unfrozen and automatically shuffled files, on Windows, are stored in '/{user}/My Documents/Max/Max for Live Devices/(patchfilename}/" for Max 6; and in '/{user}/My Documents/Max7/Max for Live Devices/(patchfilename}/" in Max 7. The files are deposited there every time you unfreeze a file for edit, If the subdirectories for data, media, and other files are not there, they are added transparently. If there were files there with the same name, unfreezing the device will overwrite any that were there before, without warning. So it is wise to look to see if something got put there, and to check if there's something you don't want to lose already there. And, it should now be clearer, the usage of a directory based on the top-level patcher'sname is why the project file settings are lost if you rename the patcher file: the project file is for the patcher of the same name, so if you store the project and rename the patcher, or try to move the project file to a different directory, it stops working.

Personally I do not find project maintenance for externals very workable under these constraints, because I often want to rename files while working on a design whenever I make a major change to it. So I gave up trying to edit the project settings and figured out how to work with its default settings, whereby it creates the directories and shuffles files transparently. If you feel all that is workable for you, though, please ignore my independent solution, because mine isn't perfect either.

Finding Paths to files

My current solution has the following steps:

  1. First, I find the path to the top-level patcher by sending PATH message to thispatcher. Note, as of Max 7.01, one forum post indicates that only the top-level patcher will consistently return the absolute path, because of continual problems with bpatchers containing subpatches in other subdirectories (which may also be aliases or symbolic link directories), Max 7.01+ may decide to return only "/" for subpatchers. Although the explicit and exact behavior is not currently documented, Cycling74 has confirmed the absolute path is best calculated for the top-level patcher only.
  2. Now there is the issue where to put the media and data files. For a while I tried to keep everything in one subdirectory, but if I renamed a frozen .amxd file and opened it for edit, the invisible project that is automatically created by opening a .amxd file (with a new filename) for edit would make new copies of the file in the "..../Max for Live/{filename}" subdirectory, and the file which is opened for edit will use those files it puts there...disregarding whatever files one had originally. Hence, after several dozen times of moving things back into one directory again, I gave up and let it have its way. Now I have /data/ and /media/ and /patchers/ sub subdirectories, and I am learning to like them.
  3. Now if one has everything organized that way, theoretically, and one does not rename the top-level patch, then one can add the files to the project file, save it, and forget it. Then, theoretically, the project will include the externals in the .amxd, munging the paths to them correctly after it reorganizes to its liking. But I don't want to include some data. For example, other people cant change a preset if pattrstorage data is locked inside the .amxd file. Also, some media files I want to share among multiple apps. So, having let the project move the files into these subdirectories when I unfreeze something, I don't actually want to include them in the project anyway. Nonethless, I now need to set the path for .JSON files, .wav data, etc, to the appropriate subdirectory; change the directory separator from "/"to "\" when moving from a Mac to a PC; and account for absolute path changes when moving between Macs and PCs and between different users.
  4. So I made a little subpatch to add paths to the /data/ and /media/ subdirectories and don't add them to the project. Now this works until you open a M4L file for edit, at which time it returns no more than a forward slash; until you freeze it, at which time it returns the full absolute path...but you have to freeze it first, after which you can't edit it. Thus, for the patcher to find data and media while in edit mode, you have to keep it in the same place, parse the path output, and when the thispatcher object returns no more than a slash, your patch can substitute the actual path to where you store your files, which all depends where you like, and can't bestandardized.
  5. Therefore, this is the addition patch, without further ado. Now you may understand better than me why it should be different, but you can't know how glad I was to get something working after all that.
Path Generation for M4L externals
Path Generation for M4L externals

It could be done more efficiently with regular expressions, but my experience has been that typical Max users actually find Max objects more understandable than a regular expression, so I did it this way.

Now there are two more little steps:

  1. Add the filename to the path: Most file systems are not happy if filenames begin with a space, so you probably want to use an "sprintf symout %s/myfile.ext' object, so there is no space between the path and filename.
  2. Add the file read command: to open a .JSON file in pattrstorage, note that you can't send pattrstorage the path string as a symbol. So you must add the READ command to a message string created by a 'fromsymbol'object. Again, the string may contains spaces, which can't be escaped. So, do that with a 'prepend read" object, or similar join, rather than use the 'read $1' messagebox shortcut.

Now your files can access externals in the automated directory structure that the M4L project file makes.

download

Well, you can just copy and paste this :). Also, if you want to test it in a working app, it is in Godel 3 for Live. But I should warn you, as of Max 7.01 and M4L 6.1.9 64bit, it requires several gigabytes of memory, for which the OS alone may need about 30 minutes to allocate to the project; and unfreezing with the 32-bit version may crash if you dont close all your other apps first.

Hide LIne Numbers
  1. ----------begin_max5_patcher----------
  2. 1949.3oc4bkziaaCE97Df7efPo8l6TtSpFjCsWROzETjSEIAAxRb7v.YICI4
  3. YIA8+d4hkFaOwxzyXKai3CVzThju2Gea7QQ+0W9hKhFWdmpNB7Kf2Ct3huZp
  4. 4BWc1ZtnshKhllbWZdRs6AiRKmNUUzDMZwMaT203twrjlqAMkfrjlDP87wY5
  5. JUZSY08cOZcy84J2y1U03Iok4kUdR.doTfnH7HSoXFChn1RbbLGapCcID7w1
  6. 1oyb8S43O+SHbWuYHgzq0ES9jcj88IO9R3HfjJsWPLn8BFtbWULeptHW03XO
  7. zR0VNuosZnq1+6kuvd0bYz9CtlpxzCKdg5CuvD4wEvJT2ZnxGiWMfw.Kl0G5
  8. rDWJ5kIgHG2QEWhvLBmOBPcxIX7txn31p800b+LkeLhFmTLIZjehNB7w8Nfj
  9. quQcYy055L0M5TUf3BFEBvfvNg.xSCSHaESzVMB+kC.xXAEGyop5t4UkEMEI
  10. S8.yuVoSxCDvj8hWXmXCRH6Dj3OWAIKkVq+hihbyCaBLsHXzCCRcxMs24U+k
  11. 8dq.CfnWa+5U+i8aUQlulmA3OUUWmLQ8Xz+xK+f4SfnKB1G7ZzOMXIm3tXfZ
  12. Vu.K9aCrnMCfGF1+mCj2Y8JYIbxQbL8zi22jdWkYTTyLOEHTHf2qqalk0IBG
  13. BvkaQsBuq1mWn9bBfBHQ.RBDl37BG9RNPkpyqAwg5ZhzGLPXbKmyfPqpfGLN
  14. GfAabKg5qAS6EAHNDfhc7N4fEvxg.EzMkIgBBr9AAWDJbD0JFPgaADH6pcwC
  15. THIIMk5ucrH+tJ+FUiNMX3gGfVBkE6UVdlvSvwhjqqaNPFQzMpJ.JTzQDhMD
  16. BwI7DeFYJchwsxr8jDTb.JXLgSAiCelXztFM6w2MDpW6OHhKbDZL+7xMT8rJ
  17. iosq.02aoCvOVaSSSnAnPBPfQPcqVDQvOM+RCanpOBNbogIT7ne2zLuGJI2m
  18. iAzYIdXEO1g0w0aN3PvXuDhewxD1oDh36rPsMzuoAetFgtPzIv1u2wLm0Vca
  19. lCczx.o.rCy385DAgYdc.gWGfcNnCz.xAiCM8hj.VxFF6rER466H0sNIbQqe
  20. .PgIIMJ.NTXneEBjyWIA48LHomZNK2XRbBl+6M8UnEQXxXmQ4uJzHs48lXzX
  21. xIKmuQKhpbvs5hrxaqCEBDA.Ada.B19V1+PtZ8cImEbV.l.nXx4WNKp.yxSZ
  22. tprZZnPAY6xCHuKQ91hJDdRXL30en5CElGELwL92W2nl9XLIWWnRKm6iTAGH
  23. R0a7hDn2iIm5iWzGOE+z17wNjgKNLfnk4T72oY3hhCYA3dWKe+kgKVHomf4c
  24. 4d9jgK8UfePidyanBPy0pBPLFnxqU1J2hglmknFQ1OX5jtXLgELi4awLzJxT
  25. WkjpBwNd34G6jOuhzfLqAiOuxq3Nke.Q+6jm22ubwxhNjIHv2kNUl0e+tbDl
  26. 8Fqw00kyqRaQs18lGrDwkopazEIM5xhkdJ7pOz05rLUwJzYltNYbtJaKuWP6
  27. Y5Jd3oKVHzEiM7DlQpDf15DI83PXaGwFb5BGBYgFdxhDjBoX3ILZPBX7iCgs
  28. cDa3sUfCxVA4DktFdKEqK6rgIxi.fINUkvhChvFdiqj8iw0o5rYkl0xtH7FB
  29. 08dAIhQ1KHpr6WKMT6aNQFjxBbvg3fr5ID6LFy8XL1mdF+Vf690gCioAErG+
  30. HfwgEjvvqech5Y4D0wBKL3RN3DVPN7PviBcs0P83CuXO+Dc0p7SzUqxCxOLe
  31. 3WLAmch5U0QXaUzmdDDwBJFTt73PXaEwDGgoxfhbSL7KwW.OUILTHSkHwwgv
  32. 1tv+vSWjfbThONDVXhXnAkvhOLqZwtkX10or3EScwpVH3C4pVvAITdDTVBQl
  33. b38SHCxMwyaMKK1QjjYytQUUunq8jTzzjO6OS0hQ9eqK7+12yQUpazsMwuqL
  34. QIUoWqaToMyq7aGzcxEGLinokYpph4Z2SapwBJ1g1s0QquaLOblTmmoKeWSR
  35. y75O8mph4K1HICXbUx77l0fvwStRmm2cVvu3gacQT6leE4u8n15W4jiiioHj
  36. zddwIHhYM7tRlBL1xSEKZDpsUTFMF5Nu4TNVPYtRRLgQrkfq2N7CiFDE6GCX
  37. rDR8kLUgV8bpav0hI98WCKfc6pmQTppbVYUqDgowwOzh4MkSpRxzK1KL+De6
  38. LeK56DGqLOxi1.unoF3U20xUEg6ld9i4c6C5FlRRRSMc+JfLAavKGuRYF91U
  39. R3qakSmuJWMc81JjbtvAvRHRJ8kLcyiNZ+KumsFUk+dlp.7tjhZv6TS0iKyy
  40. V5LBu1ee.wPJR3HPYLiJD9w.G6GWFeoCWrgDSsv+pzHFx8rEhKvdY.Iz8+Pv
  41. JjncCmcM7S5Bq1mpStfDiQN4BbmjEVHoj0jKVU1kJXHhiDwwDCk6aOxPKq0p
  42. fzR5jc1jhRnhtqppH6TPjBHSJ8kdLqstpBuURga+H7k7c04glh+kM32xmq5d
  43. c42AMFjfS8Lcm.QrspVTa+SousRoJdBjJzoiPre7Z.1Y7CFU9up77xaeBjoz
  44. nY3.TgDIQNUTlQYm8Mrhz99T7d2aTAKFBk6YVY9zwpp2ZrI8DXDqT.zoHxLF
  45. MXNdBsrh3SkRsszb4+czwYkH
  46. -----------end_max5_patcher-----------