Apart from the actual data, for each output record, 16 bytes of metadata are kept and used for sorting, partitioning, and indexing.
Metadata for tables and partitions is kept in the Hive Metastore.
Content is kept for three days, metadata for 30 days.
Considering the sensitivity of the information that is kept, it is not unreasonable to have a formal penalties regime introduced if metadata is inappropriately accessed by public officials.
The size of the metadata is small (i.e., 16 bytes per record); however, the percentage of the metadata that can be kept in the buffer is important as it affects the number of local files that will be created.
Under the revamped rule, call metadata records would be kept by phone companies, not the government.
Similarly, the maximum metadata size in bytes that can be kept in the buffer is defined with the following formula: begin{array}{l} text{maxMetaSize}_{j} = text{metaPercent}_{j} times text{mBuffer}_{j} times text{bThr}_{j} end{array} (19).
Given how many inferences about someone's personal life can be drawn from their communications metadata, you'd hope it was being kept safe away from prying eyes.
Recordings can be kept for a month, and metadata for five years.
