Optimize storage for IcuLocaleData data #45129

Activity

1. Dotnet-GitSync-Bot

   added

   area-System.Globalization

   untriagedNew issue has not been triaged by the area ownerNew issue has not been triaged by the area owner

   on Nov 23, 2020
3. marek-safar

   added

   linkable-frameworkIssues associated with delivering a linker friendly frameworkIssues associated with delivering a linker friendly framework

   on Nov 23, 2020
4. tarekgh commented on Nov 23, 2020

   

   tarekgh

   on Nov 23, 2020

   Last edited by tarekgh

   Member

   More actions

   [edited comment]

   @marek-safar do you suggest we compress it and expand it at runtime? I think we can store it as ASCII encoding or store it as compressed binary and expand it at runtime.
5. tarekgh

   removed

   untriagedNew issue has not been triaged by the area ownerNew issue has not been triaged by the area owner

   on Nov 23, 2020
6. tarekgh

   added this to the 6.0.0 milestone on Nov 23, 2020
7. krwq commented on Nov 24, 2020

   

   krwq

   on Nov 24, 2020

   Last edited by krwq

   Member

   More actions

   @tarekgh I think suggestion might be to use UTF-8 or ASCII rather than UTF-16, that should remove almost half of the bytes
8. marek-safar commented on Nov 24, 2020

   

   marek-safar

   on Nov 24, 2020

   ContributorAuthor

   More actions

   do you suggest we compress it and expand it at runtime

   Yeah, story it more efficiently so we don't have a massive string in metadata and on a whole copy of it on the heap. Most apps will use a few lcid strings only anyway. There are numerous way how to do it one of them could be to replace s_lcids, s_localeNamesIndices arrays with something like

   static ReadOnlySpan<byte>values => new byte[] {
      // "az" or "az-cyrl" or "az-cyrl-az" 
      (byte)'a', (byte)'z', (byte)'-', (byte)'c', (byte)'y', (byte)'r', (byte)'l', (byte)'-', (byte)'a', (byte)'z', 
   };
   
   // LCID values don't match the real ones in this example
   static readonly int[] s_lcids = new int[]
   {
      // 16bit LCID | index into string values | string value length
      0x10 | 0 << 12 | 7 << 4, // az-cyrl
      0xbb | 0 << 12 | 2 << 4, // az
   };

   There is a dependency from c_localeNames but that could be improved as well

   👍React with 👍2Reacted by Tarek Mahmoud Sayed and Santiago Fernandez Madero
9. marek-safar

   added a commit that references this issue on Nov 28, 2020

   Optimize storage of icu locale data

   ...

   This issue will close once commit a9ea6e3 is merged into the 'main' branch.

   a9ea6e3
10. marek-safar

    closed this as completedin #45296on Nov 30, 2020
11. marek-safar

    added a commit that references this issue on Nov 30, 2020

    Optimize storage of icu locale data (#45296)

    ...

    This issue will close once commit e039b92 is merged into the 'main' branch.

    e039b92
12. ghost locked as resolved and limited conversation to collaborators on Dec 30, 2020