If a local file is being opened, now the function tries to convert
the path into a URI. This seems to be a more reliable way to open file
on Windows 10 that has a more strict permission policy on cmd.exe.
Previously the CRC error was quite annoying to recover because
the user needed to manually delete the corrupted file from the
staging folder (without knowing the exact path of the file to
remove).
Now the IDE tries autonomously to resolve the situation by
removing the file and downloading it again.
Fixes#5394#4303
This regression originates from:
8725bb1e Clean up sketch loading
before this commit the sketch name sanitization was made on the sketch
name without the extension.
After 8725bb1e instead the name sanitization is made on the filename, so
including the ".ino" extension.
This lead to a weird corner case, caused by the limit of 63 characters
on the sketch name: before 8725bb1e it would be possible to save a sketch
with a name of exactly 63 characters, but after 8725bb1e this sketch will
suddenly becomes invalid becuase the 63 chars name + extension would exceed
the 63 characters limit.
This commit fix this regression.
Fix#5431
Previously, this used a hash of the sketch filename, so the same build
path would be generated for the same sketch between multiple
compilations. Now that the build path is stored, this requirement has
disappeared, so a random filename can be generated again. While here,
this commit also changes the prefix from "build" to "arduino_build_",
which makes it a bit more clear what the directory's purpose is.
Previously, everywhere where it was needed, the path was requested from
BaseNoGui. Because the path is based on a hash of the sketch filename,
every caller would get the same path for the same sketch.
However, it makes more sense to store the path used for a given sketch
inside the Sketch object. This prevents having to pass around or
regenerate the build path everywhere, and no longer requires the build
path to be deterministic (though it still is in this commit).
This allows removing some methods and constructors of which two versions
were available - one with a build path argument and one without.
Previously, callers of `SketchFile.delete()` would also call
`Sketch.removeFile()`, but letting SketchFile handle this is more
robust.
This is possible now that SketchFile keeps a reference to Sketch and
makes updating the Sketch file list less fragile.
Eventually this might be further decoupled by letting SketchFile
broadcast a "deleted" event instead.
This isn't much code, but it makes deletion more consistent with
renaming and saving with the SketchController handling the UI part and
Sketch actually doing the delete.
Now that SketchFile keeps a reference to its Sketch,
`SketchFile.renameTo()` can call `Sketch.checkNewFilename()`, so there
is no need for the renaming itself to go through Sketch.
This changes the parameter for `SketchFile.renameTo()` from File to
String, to enforce that only the filename is changed, not the directory
name.
These methods shouldn't really be in Base (or BaseNoGui, which did the
actual work), especially since there is already a
`FileUtils.recursiveDelete()` which just does the same thing. This
commit removes the code from Base and BaseNoGui and instead uses the
method from FileUtils.
There is one difference between these methods: the Base methods did not
delete files if the "compiler.save_build_files" preference was set.
However, the Base methods were only used when deleting a sketch, or
deleting an existing folder before overwriting it on save as, so this
preference didn't actually do what it was supposed to anyway, so
dropping it shouldn't be a problem.
This commits replaces a significant part of the code handling these
features. A lot of responsibilities are moved from SketchController to
Sketch, though the code involved is rewritten mostly.
Most of the handling now happens inside Sketch, including various checks
against the new filename. Basically SketchController processes the user
input to decide what needs to be done, and Sketch checks if it can be
done and does it.
If problems occur, an IOException is thrown, using a translated error
message that is shown by SketchController as-is. This might not be the
best way to transfer error messages (regular IOExceptions might contain
less-friendly messages), so this might need further improvement later.
In addition to moving around code and responsibilities, this code also
changes behaviour in some places:
- Because Sketch and SketchFile are now in control of renames and
saves, they can update their internal state after a rename. This
removes the need for reloading the entire sketch after a rename or
save as and allows `Editor.handleOpenUnchecked()` to be removed.
- When renaming the entire sketch, all files used to be saved before
renaming, since the sketch would be re-opened after renaming. Since
the re-opening no longer happens, there is no longer a need to save
the sketch, so any unsaved changes remain unsaved in the editor after
renaming the sketch.
- When renaming or adding new files, duplicate filenames are detected.
Initially, this happened case sensitively, but it was later changed to
use case insensitive matching to prevent problems on Windows (where
filenames cannot differ in just case). To prevent complexity, this
did not distinguish between systems. In commit 5fbf9621f6 (Sketch
rename: allowig a case change rename if NOT on windows), the
intention was to only do case insensitive checking on Windows, but it
effectively disabled all checking on other systems, making the check
not catch duplicate filenames at all.
With this commit, all these checks are done using `File.equals()`
instead of comparing strings, which is already aware of the case
sensitivity of the platform and should act accordingly.
- Some error messages were changed.
- When adding a file, an empty file is not created directly, but only a
SketchFile and EditorTab is added. When the sketch is saved, the file
is created.
- When importing a file that already exists (thus overwriting it),
instead of replacing the SketchFile instance, this just lets the
EditorTab reload its contents. This was broken since the introduction
of EditorTab. The file would be replaced, but not this was not
reflected in the editor, which is now fixed. This change allows
`Sketch.replaceFile()` to be removed.
- When importing a file that does not exist yet (thus adding it), a tab
is now also added for it (in addition to a SketchFile). This was
broken since the introduction of EditorTab, and would result in the
file being added, but not shown in the editor.
This commit adds a `Sketch.renameFileTo()` method, to rename a single
file within the sketch. It would be better to integrate its contents
into `Sketch.renameTo()`, but that does not have access to the `Sketch`
instance it is contained in. This will be changed in a future commit.
This makes a few related changes:
- `FileUtils.replaceExtension()` is introduced to handle replacing the
.pde extension with .ino.
- Instead of iterating .pde files on disk, this iterates SketchFiles in
memory, saving another lookup from filename -> SketchFile later.
- `SketchController.renameCodeToInoExtension()` is removed. Now it no
longer needs to look up the SketchFile and FileUtils handles the
extension replacement, this method did not have any reason to exist
anymore.
- Instead of hardcoding the .pde extension, a new
Sketch.OLD_SKETCH_EXTENSIONS constant is introduced.
Sketch already stores the sketch folder, and the sketch name should be
identical to the folder name. In the case where the filename passed to
the sketch constructor is not the primary .ino file (named after the
sketch), this will slightly change behaviour. However, the calling code
should prevent this from happening, and with the old code, some internal
assumptions were probably violated, so this changes makes handling this
situation a bit more robust.
Since the actual filename passed to Sketch is no longer used, it is no
longer required to pass the name of the primary .ino file. At some
point, the constructor should probably be changed to accept a folder
name instead of a filename, but that would require a lot of changes
to trace this back through the code, so this is something for later.
Keeping filenames as File objects for as long as possible is generally a
good idea and this removes a dependency on `Sketch.getMainFilePath()`,
so it can be removed later.
Previously, it returned a File object, which the Sketch separately
stored from the primary SketchFile. By letting it just return the
SketchFile, and let callers query that for the filename, Sketch does not
need to store the File object itself and there is less chance of info
getting out of sync.
Instead, just the File object when requested. It is not used during
normal operation (just when adding files, or using save as), so no point
in already creating the object in the constructor.
That name more accurately reflects its purpose: It represents a single
file within a sketch. This just updates the class name and variable
names referring to these objects and some comments, so no behaviour
should change.
Previously, the index of the SketchCode instance in the list kept by
Sketch was kept, which isn't really robust.
With this change, Sketch.indexOfCode is no longer needed and is removed.
For determining if the current file was a sketch file, it previously
(indirectly) used a hardcoded "ino" comparison. Now, it uses
`SKETCH_EXTENSIONS` so it also applies to .pde files and the hardcoded
"ino" (and the methods leading up to it) can be removed.
This lets it use FileUtils.splitFilename and reference Sketch.EXTENSIONS
and the new Sketch.DEFAULT_SKETCH_EXTENSION directly, allowing to remove
a few helper functions.
Before, `getPrettyName()` would return the extension-less name for all
files. There were a lot of places that checked for .ino and/or .pde
files and and called `getPrettyName()` for those, and `getFileName()`
for others. By moving this check into `getPrettyName()`, all those
callers become more simple, and more consistent (there were 5 different
checks to basically achieve the same thing).
There are small changes in behaviour, where .pde is now also hidden but
was not before. Also, the print header now shows extensions for other
files, which makes it more consistent with the tab names. For cases
where the old behaviour was still required, `Sketch.getBaseName()` was
added.
At the same time, the actual handling of the filenames is simplified by
using methods from FileUtils.
With this change `Sketch.getFileNameWithExtensionIfNotIno()` and
`SketchController.getHiddenExtensions()` are no longer needed and are
removed.
Sketch is now called SketchController, since it didn't really represent
a sketch, but just handled the GUI-related stuff for a given sketch
(note that it is not strictly a controller in the MVC-sense, but it does
have a similar function). SketchData more accurately represented the
actual sketch, so it is now called Sketch. Below, the new names are
used.
Editor now keeps both a current Sketch and SketchController object, and
the Sketch object is created by Editor and passed to SketchController,
instead passing a File and letting SketchController create the Sketch.
Wherever possible, code now uses the Sketch directly (or indirectly,
through the new `SketchController.getSketch()`) and the accessors in
SketchController that merely forwarded to Sketch have been removed.
There are few things that now live in SketchController but should be
moved to Sketch (`isModified()`, `isUntitled()`), so some of the code
still has a dependency on SketchController that should be removed later.
This commit mostly renames classes, methods and variables, it should not
change the behaviour in any way.
When adding a file to a sketch (using drag and drop, or the Sketch ->
Add file... menu item), .o, .a and .so files would be saved into a
"code" subdirectory of the sketch. This seems to be a remnant of
processing, where also .dll and .jar files could be added to a sketch to
be used. In the Arduino IDE, these code files serve no special purpose,
and are not treated specially, so it makes no sense to keep this code
around.
One implication of this is that when "save as" is used, a "code"
subdirectory is no longer copied, which might affect people using this
"code" subdirectory for other purposes.
Similarly, there is support for a "data" subdirectory, in which all
other files (that are not sketch source files) are stored, and which is
also copied on "save as". Support for this folder is kept intact, since
this appears occasionally used (the ESP8266 project uses it to store and
upload additional data files, for example).
This change was discussed on the mailing list in the "Anyone using
"data" and "code" subdirectories in sketches?" thread:
https://groups.google.com/a/arduino.cc/forum/#!msg/developers/zPlraPq55ho/ejrLqITnAgAJ
By now, all calls to `addCode()` were followed by a call to
`sortCodes()`, and it seems like a task for SketchData to keep its list
sorted. Previously, this separation made some sense, since `addCode()`
was also used while loading a sketch, and you would only want to sort
once. Now, sketch loading uses a SortedSet, so this is no longer a
requirement.
With this commit, any warnings about invalid sketch filenames are not
shown when the sketch is reloaded. This reloading happens whenever the
IDE window is focused, so re-logging warnings all the time isn't really
helpful, so this hides them.
Previously, the Sketch constructor called its `load()` function, which
called the `SketchData.load()` function to load files and then
`Editor.sketchLoaded()` to initialize the GUI with the loaded files.
When external editing was enabled, `Sketch.load()` was called again
when activating the Arduino app, to reload the entire sketch.
With this commit, the `Sketch.load()` function is removed, and
`SketchData.load()` is called from the SketchData constructor. Instead
of Sketch calling `Editor.sketchLoaded()`, that method is renamed
to `createTabs()` and called by `Editor.HandleOpenInternal()` directly
after creating the Sketch object.
Handling of external editor mode has also changed. When the Arduino
application is activated, instead of fully reloading the sketch (through
the now-absent `Sketch.load()` method), the new `SketchData.reload()`
method is called to reload the list of files in the sketch. If it
changed, all tabs are re-created. If not, only the current tab is
reloaded. When the user switches from one tab to another, that tab is
also reloaded. This ensures that the visible tab is always up-to-date,
without needlessly reloading all tabs all the time. When external
editing mode is enabled or disabled, all tabs are reloaded too, to make
sure they are up-to-date.
When re-creating all tabs, no attempt is made to preserve the currently
selected tab. Since adding or removing files happens rarely, this should
not be a problem. When files are changed, the currently selected tab is
implicitly preserved (because the tab is reloaded, not recreated). The
caret (and thus scroll) position is preserved by temporarily changing
the caret update policy, so the caret does not move while the text is
swapped out. This happens in `EditorTab.setText()` now, so other callers
can also profit from it.
To support checking for a changed list of files in
`SketchData.reload()`, a `SketchCode.equals()` method is added, that
just checks if the filenames are equal. Additionally, the loading of the
file list for a sketch has now moved from `SketchData.load()` to
`SketchData.listSketchFiles()`, so `reload()` can also use it. At the
same time, this loading is greatly simplified by using a sorted Set and
`FileUtils.listFiles()`.
In external editor mode, to ensure that during compilation the version
from disk is always used instead of the in-memory version, EditorTab
detaches itself from its SketchCode, so SketchCode has no access to the
(possibly outdated) in-memory contents of the file.
Instead of manually sorting the primary file at the start, and fiddling
to keep it there during resorting, this just modifies the sorting
comparator used to sort any primary files at the start. This is slightly
more generic than needed, also supporting multiple primary files, to at
least not break the Comparator preconditions when for some reason there
are multiple primary files.
These used to iterate over the list of SketchCodes to find the right
one, and if so, let the List do the same again to remove it or find the
index. This can be simplified to just let list take care of things
instead.
Technically, there is a small difference, since `List.remove()` and
`List.indexOf()` will check using `equals()`, while the original code
used `==`, but these should be effectively the same here. Also, the
original code first used `==` to see if the object was present and then
let List find it again using `equals()`, so that was a bit inconsistent
anyway.
This makes checking for the primary file easier, without having to know
the index of a file in the list of tabs, or relying on the fact that the
primary file is always first (it still is, though).
This changes some places in Sketch to use the new
`SketchCode.isPrimary()` method, but there probably are a lot more
places in the code that could be start to use it as well.
It was not used, and since it only updated the `name` attribute, but not
the corresponding `file` attribute, nor actually handled renaming actual
files, having this method around would actually be harmful, so just drop
it.
This class served no purpose anymore, so it can be removed. The
`SketchCode.getMetadata()` and `setMetaData()` methods only served to
keep track of a SketchCodeDocument instance (and were no longer used),
so these are removed too, just like some SketchCode constructors dealing
with this metadata object.
Now that each file in the sketch has its own text area in the GUI, it is
no longer needed to store the (possibly modified) contents of each file
inside SketchCode. Keeping the contents in the text area is sufficient.
Doing so allows removing the code that dealt with copying contents from
the text area into the SketchCode instance at the right time, which was
fragile and messy.
However, when compiling a sketch, the current (modified) file contents
still should be used. To allow this, the TextStorage interface is
introduced. This is a simple interface implemented by EditorTab, that
allows the SketchCode class to query the GUI for the current contents.
By using an interface, there is no direct dependency on the GUI code. If
no TextStorage instance is attached to a SketchCode, it will just assume
that the contents are always unmodified and the contents from the file
will be used during compilation.
When not using the GUI (e.g. just compiling something from the
commandline), there is no need to load the file contents from disk at
all, the filenames just have to be passed to arduino-builder and the
compiler. So, the SketchCode constructor no longer calls its `load()`
function, leaving this to the GUI code to call when appropriate. This
also modifies the `SketchCode.load()` function to return the loaded
text, instead of storing it internally.
To still support adding new files to a sketch (whose file does not
exist on disk yet), the EditorTab constructor now allows an initial
contents to be passed in, to be used instead of loading from disk. Only
the empty string is passed for new files now, but this could also be
used for the bare minimum contents of a new sketch later (which is now
down by creating a .ino file in a temporary directory).
Another side effect of this change is that all changes to the contents
now happen through the text area, which keeps track of modifications
already. This allows removing all manual calls to `Sketch.setModified()`
(even more, the entire function is removed, making `Sketch.isModified()`
always check the modification status of the contained files).
This fix a regression introduced in:
048a8a61 (VersionHelper now correctly strip snapshot info)
actually neither 048a8a61 nor the version before are correct becuase:
048a8a61 - strips all the extra `-snapshot` and `+build`
previous - doesn't handle the case `x.y-snapshot`
Now both are handled correctly and a test has been added to verify this.
To be completely semver compliant we should deny versions in the
format `x.y`, but this will break all legacy version that have been
published until now, so this changed should be postponed for the next
major release of the IDE.
Fix#5251
There is no reason to bundle this file.
If the index file is not available an empty index is
returned by the parser.
Fix#5143
(together with e80c08: Use a specific hardware/package_index_bundled.json)
Previously if a 3rd party core would require a tool already bundled
in the IDE then boards manager skipped the installation of that tool.
This is could lead to missing tools if the IDE is upgraded and the
bundled tools may change.
This patch fixes the bug by always installing tools when needed, even
if they are already bundled.
This covers a very convoluted use-case that may be reproduce this way:
1. Using an previous version of the IDE, a new AVR core is installed
using the board manager.
2. The IDE is then updated so the core installed in 1. is now also the
bundled one
3. The AVR core installed 1. is now removed using the board manager
4. The board manager will uninstall the (presumably) no longer used tools,
from the built-in folder leaving, in fact, the IDE without the
bundled tools that are supposed to be read-only.
This commit fix this bug by actually making the built-in tool read-only
Previously, during the build, the full package_index.json was downloaded
and distributed with the Arduino IDE.
This lead to a situation where it was difficult to test new AVR cores
before publishing them to the public package_index.json.
Now the bundled AVR core is specificed in the file:
`hardware/package_index_bundled.json`
this index is loaded from the IDE at startup and the package_index.json
is overlayed on it.
This should also solve part of #5143 (Repeatable builds and snapshots of
package/library indexes)
Martino Facchin
Cristian Maglie
Kyle Evans
Matthijs Kooijman