Talk:Placename Format
From World Wind Wiki
| Revision as of 20:47, 8 February 2005 (edit) 80.131.143.224 (Talk) ← Previous diff |
Current revision (02:32, 9 February 2005) (edit) (undo) Llynix (Talk | contribs) |
||
| Line 29: | Line 29: | ||
| file# along with the seek offset with each entry. | file# along with the seek offset with each entry. | ||
| (gillesk, 08-feb-05) | (gillesk, 08-feb-05) | ||
| + | |||
| + | OK.. first thing I purpose is a header for meta data descriptions. Not the actual meta data itself, but for the data names such as County, State, Feature Type, etc. This would eliminate a lot of redundency in the files. We don't lose much flexibility this way, but a particular placename file would be limited to certain meta data types. This isn't too much of a problem, as different placenames should be placed in different files anyway. Second thing I purpose is something that defines the string format. Currently I'm not sure how they are encoded / read by worldwind but ASCII or unicode characters do not work at all. Even having ASCII support would allow for special chars. I would recommend an ASCII format by default, and then other encoding options available via the header. -- Llynix 08-02-05 | ||
| + | |||
| + | ------------------- | ||
| + | Has anyone tried the BinarySerializer in .NET? There had been issues of it being inefficient, dont remember if it was space or read/writing speed. It will handle Unicode. There is also SharpZip lib if we are really worried about space. -- adamhill (?) | ||
Current revision
The first thing that could be done here IMHO is get rid of the "XML-inspired" way metadata is stored. This implements a flexibility (be able to store _any_ attribute not known in advance) that is not currently needed at all (those placenames that have metadata currently have the exact same set AFAICT), and I doubt will be needed with future datasets. My suggestion here would be to (conceptually) split metadata in two categories: fixed, and variable. The fixed metadata, like name, lat, lon, is always there and implicitly known as currently is the case.
Variable metadata names are defined in a header section of the file as suggested, and each record has a bit vector (e.g. the int currently used for metadata count, would enable to code 16 metadata attributes) that indicates what metadata field (in a well-defined order) is present. This way, we could keep the metadata flexibility (with md either lacking or only partly present), along with great time and space savings.
- number of files. For static data like this, a "tile-oriented"
structure is not needed (the grouping can be done inside the file
along with an offset table), and slows down some operations. For
example, the several second "pop-up" delay the placefinder had a few
iterations ago was due to the constructor initializing a "placecount"
per layer. There are only 23 layers, but 263 total files. Opening each
file, reading an int and closing it again amounted to a quite
noticeable amount of time. (That is now deferred to search time, but
it is still there). Having only one file per layer would also reduce
the size of the index files by 30%, as there is no need to store the
file# along with the seek offset with each entry.
(gillesk, 08-feb-05)
OK.. first thing I purpose is a header for meta data descriptions. Not the actual meta data itself, but for the data names such as County, State, Feature Type, etc. This would eliminate a lot of redundency in the files. We don't lose much flexibility this way, but a particular placename file would be limited to certain meta data types. This isn't too much of a problem, as different placenames should be placed in different files anyway. Second thing I purpose is something that defines the string format. Currently I'm not sure how they are encoded / read by worldwind but ASCII or unicode characters do not work at all. Even having ASCII support would allow for special chars. I would recommend an ASCII format by default, and then other encoding options available via the header. -- Llynix 08-02-05
Has anyone tried the BinarySerializer in .NET? There had been issues of it being inefficient, dont remember if it was space or read/writing speed. It will handle Unicode. There is also SharpZip lib if we are really worried about space. -- adamhill (?)
