Add ability to drag files to and from external applications#7849
Add ability to drag files to and from external applications#7849AW1534 wants to merge 72 commits intoLMMS:masterfrom
Conversation
|
To hell with CodeFactor. |
|
once #7627 is merged, I'd like to add the ability to drag sample clips out of LMMS too. That could be for a different PR though. |
|
I tested this briefly, and it seems to work! I'm using nautilus on GNOME to drag files in. Things I noticed (these aren't that important though):
|
Okay, this should be a relatively simple fix.
Thats actually not in master, it's is a regression- Nice catch!
This is out of scope as it isn't in master, but it's definitely a good idea for another PR. |
|
I tested this again, and those two issues do appear to be fixed! |
|
In Windows MSVC, dragging instruments to the Song-Editor doesn't work 2025-04-18.23-03-41.mp4 |
szeli1
left a comment
There was a problem hiding this comment.
I would like a few things changed.
Also I'm testing this and When I drag a sample clip into AudioFileProcessor, it doesn't load the audio for me. Firstly create a sample clip and load audio, then create an empty AudioFileProcessor and try to drag the clip into the processor.
Dragging in files from outside works nicely.
What else to test?
huh, I don't know why I'm just now seeing this comment, sorry. Master doesn't let me drag sample clips into AFP either. In fact, I think master behaves worse, as it accepts the drop, but just has weird behaviour. So I think this is out of scope. However, I was planning to add that once your sample exporting PR is merged. |
szeli1
left a comment
There was a problem hiding this comment.
I like the code changes and I believe the recent commits made the code easier to implement in the future which is nice. I didn't find any issues in the code.
I tested dragging samples from outside (ogg, flac, mp3) and it worked. However when I opened the file browser and dragged an instrument in, it didn't get added to the song editor. I couldn't test vestige because I could not load the instrument. I also tried to link a knob to an automation clip and it did not accept it.
|
@szeli1 fixed. The issue was being caused by the |
|
@szeli1 any idea why the builds might be failing on windows? |
Add |
|
@szeli1 Thanks, that worked! This should be ready for merge now. |
szeli1
left a comment
There was a problem hiding this comment.
I tested this again and everything worked as expected. I was able to drag in sound files from outside, vsts and project files worked as well.
The reason I'm not approving this is I found that snapping the lmms window to the screen's right side (for example) did not work after this change. This worked on master. I use Linux mint, I built.
Steps to reproduce:
- launch lmms
- grab the widow
- drag it until the cursor reaches one side of the screen
- normally, the window would resize to cover half of the screen
- currently it does nothing
This PR is a big improvement, so I hope to get this merged.
|
this is a really weird issue, but I see no reason why my PR would cause this. Are you using a custom build to test master and AppImage to test my PR? if that's the case, you should test master with the AppImage to see if you get the same issue. I am unable to reproduce this on the AppImage or custom build |
|
@szeli1 do the keyboard shortcuts work? (super + right) |
I believe it didn't work for me when I tested it. |
Wait this happened to me in an independent branch, it seems like this is an unrelated bug |
szeli1
left a comment
There was a problem hiding this comment.
I tested this PR and I approve it.
Co-authored-by: Dalton Messmer <messmer.dalton@gmail.com>
- change `mimetypes` type to `std::map<std::string, std::set<std::string>> mimetypes` - remove lv2 and xiz from hardcoded mimetypes map
Co-authored-by: szeli1 <143485814+szeli1@users.noreply.github.com>
Co-authored-by: Dalton Messmer <messmer.dalton@gmail.com>
messmerd
left a comment
There was a problem hiding this comment.
Haven't tested it, but the code changes look fine to me.
Co-authored-by: Dalton Messmer <messmer.dalton@gmail.com>
szeli1
left a comment
There was a problem hiding this comment.
I like decodeMimeData and processDragEnterEvent, they are a huge improvement to the code.
| * The function also uses fallback decoding via StringPairDrag in case the type and value | ||
| * were encoded in a non-file-based drag operation. | ||
| * | ||
| * @param _de Pointer to the QDropEvent containing drag-and-drop data. |
There was a problem hiding this comment.
I can't find "_de" as a param.
| const char * supportedFileTypes; //!< csv list of extensions | ||
| const PixmapLoader* logo; | ||
| const char* supportedFileTypes; //!< csv list of extensions | ||
| const char* supportedMimetype; //!< mimetype supported by the plugin. Should be null if no filetypes are supported |
There was a problem hiding this comment.
Should be csv maybe? But currently we don't need multiple new keys for one plugin.
| ",mp3" | ||
| #endif | ||
| , | ||
| "samplefile", |
There was a problem hiding this comment.
Is "samplefile" only used by AudioFileProcessor? I believe other widgets send audio data with an other key already.
Removing the "samplefile" key may be outside of this PR's concerns.
There was a problem hiding this comment.
Also patman uses "samplefile" even though it doesn't define it.
| QString decodeKey( const QMimeData * mimeData ); | ||
| QString decodeValue( const QMimeData * mimeData ); |
| //! use QString Clipboard::decodeKey(QMimeData) instead | ||
| static QString decodeKey(QDropEvent* de); | ||
| //! use QString Clipboard::decodeValue(QMimeData) instead | ||
| static QString decodeValue(QDropEvent* de); |
| //! gets the extension of a file, or returns the string back if no extension is found | ||
| static QString getExtension(const QString& file) | ||
| { | ||
| QFileInfo fi(file); | ||
| const QString ext = fi.suffix().toLower(); | ||
| return ext.isEmpty() ? file.toLower() : ext; | ||
| } |
There was a problem hiding this comment.
Please remove this. QT already has this functionality (QFileInfo) and lmms also implements this internally in some utility function.
| if (it == mimetypes.end()) { return false; } | ||
|
|
||
| const auto& fileTypes = it->second; | ||
| const auto extStr = getExtension(ext).toStdString(); |
There was a problem hiding this comment.
| const auto extStr = getExtension(ext).toStdString(); | |
| const auto extensionStr = getExtension(extension).toStdString(); |
| bool isMidiFile(const QString& ext) { return isType(ext, "midifile"); } | ||
| bool isVstPluginFile(const QString& ext) { return isType(ext, "vstpluginfile"); } | ||
|
|
||
| //! Gets the clipboard mimedata. NOT the drag mimedata |
There was a problem hiding this comment.
Move the comment into the header.
| const QList<QUrl> urls = mimeData->urls(); | ||
|
|
||
| QString type{"missing_type"}; | ||
| QString value{}; |
There was a problem hiding this comment.
| QString value{}; | |
| QString value; |
@messmerd suggested this change. It was incorrect to assume value isn't initialized when defined. This isn't the case with basic c++ types such as float and int.
You've approved this PR while plugin keys are still present in the code.
This did happen. I think it is nice that plugins define the extensions, so this is a good change.
The existence of this code makes the changes not really "plugin-specific" bool isAudioFile(const QString& ext) { return isType(ext, "samplefile"); }
bool isProjectFile(const QString& ext) { return isType(ext, "projectfile"); }
bool isPluginPresetFile(const QString& ext) { return isType(ext, "pluginpresetfile"); }
bool isTrackPresetFile(const QString& ext) { return isType(ext, "trackpresetfile"); }
bool isSoundFontFile(const QString& ext) { return isType(ext, "soundfontfile"); }
bool isPatchFile(const QString& ext) { return isType(ext, "patchfile"); }
bool isMidiFile(const QString& ext) { return isType(ext, "midifile"); }
bool isVstPluginFile(const QString& ext) { return isType(ext, "vstpluginfile"); }
Usually file types need human readable names, so I think marking them with a number complicates things.
Keys are still hardcoded into the application, only extensions differ which is an improvement, but doesn't achieve the main goal, it doesn't truly fix the problem.
This didn't happen. My main issue is that I would like to replace string keys with enum keys. This means keys like "vstpluginfile" must be defined for example in clipboard, so the design mistakes are kept. You've approved this PR @messmerd while these design mistakes are still present. |
Co-authored-by: szeli1 <143485814+szeli1@users.noreply.github.com>
allejok96
left a comment
There was a problem hiding this comment.
I didn't know LMMS' drag and drop system was such a horrifying mess. I like what you have done to clean up processDragEnterEvent. But there's more cleanup that needs to be done in the drag n drop code and now is the time to rethink that, before building more things on top of the old code.
| return {type, value}; | ||
| } | ||
|
|
||
| void startFileDrag(gui::FileItem* f, QObject* qo) |
There was a problem hiding this comment.
Why did you move this code here? It's only used by FileBrowser and requires the gui namespace.
There was a problem hiding this comment.
I don't remember this, but I could be mistaken. I agree with @allejok96
There was a problem hiding this comment.
Yes, this review comment. I'll revert that when I get the chance.
|
|
||
| const QMimeData * getMimeData() | ||
| // there are other mimetypes, such as "samplefile", "patchfile" and "vstplugin" but they are generated dynamically. | ||
| static std::map<std::string, std::set<std::string>> mimetypes = |
There was a problem hiding this comment.
I strongly dislike the use of this mimetype map and I don't understand what benefit it brings.
First of all, these are not mimetypes, they are just arbitrary hard-coded and bug-prone strings.
Secondly, mimetypes are used to indicate the data format. But .xiz and .lv2 gets registered under the the same pseudo mimetype called "pluginpresetfile" which makes it possible to drag a .lv2 files into ZynaddsubFX. Mimetypes should be as specific as possible. This design removes useful information.
Thirdly, if a new plugin was created that could open .xyz files, it's not enough to simply define .xyz in the list of supported extensions, it now also has to have a pseudo mimetype. If we want to guarantee that the .xyz file cannot be opened with any other plugin we need to define yet another pseudo mimetype and corresponding isXyzFile() function to be able to call decodeMimeData(). But this is Clipboard and it should not even know about all these plugin specific formats. Like the comment says they should be generated dynamically.
Now to be fair @AW1534 you didn't create this pseudo mimetype system, it was already in place. But it feels like we should be moving to something more standardized instead of creating an every harder requirement of using this bad system.
Possible solution: instead of dealing with generalized file types, return the exact file type and let each caller check that against a list of file types that it supports. For common things like "samplefile" the list could be a constant in Sample.h etc.
There was a problem hiding this comment.
I strongly dislike the use of this mimetype map and I don't understand what benefit it brings.
It was an attempt to move away from hard coded types. This attempt failed as things stand now although it partly improved the code because now plugins can define their supported file types (so the hard coded extensions are moved out of clipboard).
Secondly, mimetypes are used to indicate the data format. But .xiz and .lv2 gets registered under the the same pseudo mimetype called "pluginpresetfile" which makes it possible to drag a .lv2 files into ZynaddsubFX. Mimetypes should be as specific as possible.
I'm okay with grouping more extensions into 1 type, if they are converted to the same kind of data (for example .flac and .ogg are both audio data), this conversion should happen as soon as possible.
But this is Clipboard and it should not even know about all these plugin specific formats. Like the comment says they should be generated dynamically.
I don't know if it is possible to generate this dynamically, or if it is worth the cost of complexity. Currently gui assigns functionality to identifiers like samplefile. Therefor if we would dynamically define these identifiers, then we couldn't use them compile time. If we would declare something compile time that would replace them, then we end up in the same hard coded situation.
Possible solution: instead of dealing with generalized file types, return the exact file type and let each caller check that against a list of file types that it supports. For common things like "samplefile" the list could be a constant in Sample.h etc.
StringPairDrag::processDragEnterEvent(_dee, {"trackpresetfile", "pluginpresetfile", "samplefile", "instrument",
"midifile", "soundfontfile","patchfile","vstpluginfile","projectfile",
QString("track_%1").arg(static_cast<int>(Track::Type::Instrument)), QString("track_%1").arg(static_cast<int>(Track::Type::Sample))});Here we would need to know about vstpluginfile, we could ask vst plugin for it's extensions, but then we would need to know that we are asking specifically vst plugin and not lv2 plugin for example. Therefor vst plugin is ought to be known compile time in some way.
I would suggest a simple hard coded approach where we define an enum containing all types. We would use this enum in gui, to replace strings (benefit: somebody can't misspell a type accidentally like in vst plugin's case). This solution isn't clean, but it is simple. We could still define extensions in plugins, then loop trough them to convert extensions to keys.
There was a problem hiding this comment.
@AW1534 having taken a second look at the code, I realize that the only distinction between pluginpresetfile, vstpluginfile, soundfontfile and patchfile is which type of cursor icon they get. Otherwise they are all treated the same - the correct plugin is loaded and Plugin::loadFile() is called. Is it worth having different names and functions and types for the sake of a cursor icon? If all of these were loaded as pluginpresetfile, then by the grace of InstrumentTrackWindow SF2 player and Patman would gain drag and drop support and you could remove VestigeInstrumentView::dropEvent.
| _de->accept(); | ||
| } | ||
| else if( StringPairDrag::decodeKey( _de ) == "sampledata" ) | ||
| else if (type == "sampledata") |
There was a problem hiding this comment.
I can't find anything that creates a "sampledata" string pair btw
There was a problem hiding this comment.
This is why I do not support using strings as keys / types. Fixing this is outside of the scope of this PR.
| QString filePath = QUrl::fromLocalFile(f->fullName()).toString(); | ||
|
|
||
| // Internal LMMS type | ||
| mimeData->setData("application/x-lmms-type", internalType.toUtf8()); |
There was a problem hiding this comment.
Is this internal data ever used? If it should be kept it should use mimeType(MimeType).
| return( QString::fromUtf8( mimeData->data( mimeType( MimeType::StringPair ) ) ).section( ':', 1, -1 ) ); | ||
| } | ||
|
|
||
| std::pair<QString, QString> decodeMimeData(const QMimeData* mimeData) |
There was a problem hiding this comment.
Looking at its usecases I think decodeMimeData should be split in two.
Clipboard::getFile({list of supported extensions}) -> ext, path
the opposite of startFileDrag
Clipboard::getStringPair() -> key, value
the opposite of copyStringPair
Even if the return types are the same, having separate functions with specific names for specific usages makes the code much more readable.
There was a problem hiding this comment.
If this was done, the caller of getFile would need to know every single supported extension. then we run into the same issues of hardcoding that this PR solved.
There was a problem hiding this comment.
Even if the return types are the same, having separate functions with specific names for specific usages makes the code much more readable.
I disagree with this suggestion.
There was a problem hiding this comment.
the caller of getFile would need to know every single supported extension
@AW1534 currently the caller always knows what file extension it expects, except in these two cases:
"samplefile"
In this case the caller relies on SampleDecoder so it doesn't need to know about file types. To get the supported extensions, the code could call SampleDecoder::supportedAudioTypes()
Actually, in your implementation "samplefile" is a dynamic mime type loaded by AudioFileProcessor. This means the AFP definition would need to be manually updated each time a new file format is added to SampleDecoder. And also, if APF is excluded from the build, any code using "samplefile" drag and drop would break.
"pluginpresetfile"
In this case the caller just want to know if there exists any plugin that supports a given file extension. This is used by TrackContainerView and InstrumentTrackWindow. Since the plugin specific mime types are loaded by PluginFactory I suggest that PluginFactory should be responsible for providing a list of file extensions that there exists a plugin for, similar to how SampleDecoder should be responsible for knowing which file types that can be loaded as samples. Then TrackContainerView could query this list of plugin extensions before it tries to load a plugin.
| else { dee->ignore(); } | ||
| } | ||
| else { dee->ignore(); } | ||
| StringPairDrag::processDragEnterEvent(dee, {"samplefile", QString("clip_%1").arg(static_cast<int>(Track::Type::Sample))}); |
There was a problem hiding this comment.
The clipboard can contain multiple types of data, right? If dragging a sample clip would create both a stringpair and a URL then Track could use the stringpair and everything else would just use the regular sample file.
I've mocked up a system of completely dynamic file types. There are no functions like |
|
What I've learned the past days is that the clipboard system builds on unspoken rules and is in need of refactoring. Maybe we could add drag and drop without having to refactor... Idk @AW1534 would know. But at the moment, adding a lot of |
forked from AW1534's work on LMMS#7849
Adds the ability to drag files in and out of LMMS. This required a bit more refactoring than I expected.
video2.mp4
I should note that this doesn't just work to and from the desktop, I've tested with firefox, CLion, dolphin and discord. It also works with more than just sample files (projects, presets, midis, vsts)