Propose new functionalities for Decidim software
#DecidimRoadmap Designing Decidim together
Is your feature request related to a problem? A clear and concise description of what the problem is.
The futureu.europa.eu platform is one of the first Decidim instances to be using the machine translation feature.
Based on the proposal “Machine translation enhancement for source language detection edge cases” we’d like to offer a more complete solution to the source language problem.
As described in the additional context, adding a dropdown for language selection on UGC (User generated content) forms might not be enough as some users won’t notice it or take the time to select the right language.
Describe the solution you'd like
We propose to leverage on the automatic language detection feature that some machine translation services offer. This way we’d be able to select the source language for the user.
The issue with automatic language detection is that it doesn’t work well on short sentences like (OK , Da etc.). Also some languages are very close (ex : Romanian and Moldavian) and errors can be made.
Considering the user experience needs to be as smooth as possible, and in order to avoid issues with availability of the language detection software, the implementation we are considering is to have a Javascript widget that would query the external service through AJAX. This will allow us to use external services like google translate, or add our own implementation (for instance a small Python script that would allow language detection - Please note there are no mature enough libraries to use in the ruby context). Also, this implementation would allow the user to select the language from a drop down in case the Confidence score provided by the used library is not high enough.
Acceptance criteria
Given that I’m logged in
When I am writing a comment / proposal / meeting in the same language as the one I selected to browse the website
Then the form displays the language selection dropdow with my preferred language.
Given that I’m logged in
When I am writing a comment / proposal / meeting in another language than the one I selected to browse the website
If the language detection is at a high confidence level
Then the language of my contribution is automatically selected and a message offers me via a click to display the language selection dropdown to correct if any mistake was made.
If the language detection is at a low confidence level
Then a message appears explaining the situation and showing the detected language and its confidence level (%).
Given I am a logged in user
When I create an event of proposal
Then the language detection should be done only by one field, the body of the content added.
Describe alternatives you've considered
None
Additional context
This builds upon the meta proposal “Machine translation enhancement for source language detection edge cases”
Resources : Libraries for automatic language selection
https://github.com/ankane/fastText < Ruby fast text implementation
https://github.com/bung87/whatlangid < Python library
https://github.com/tremend-cofe/language-detection < Python implementation of the language detection application (based on whatlangId)
Does this issue could impact on users private data?
No
Funded by
EU Commission
Report inappropriate content
Is this content inappropriate?
1 comment
I agree with the general idea of using automatic language detection, but have some doubts regarding the implementation:
1) Don't add the "We could not automatically detect...". As far as I know, this isn't a feature that other platforms with automatic translations have (Facebook and Twitter). If a translation fails, it fails. The fallback is to give the option to show the original text.
2) Regarding AJAX/Python/Ruby. As this depends heavily on which service/library you plan to use in your app, it would need to extend the contract that we already have in MyTranslationService, for instance, `MyTranslationService.detect(text)` or something like that
Add your comment
Sign in with your account or sign up to add your comment.
Loading comments ...