File upload guide - Revision history

Nils Duepont: /* Uploading "missing" or N/A values */

2024-11-29T10:51:45Z

‎Uploading "missing" or N/A values

Nils Duepont: /* Uploading "missing" or N/A values */

2024-09-12T18:28:45Z

‎Uploading "missing" or N/A values

Nils Duepont: /* Uploading "missing" or N/A values */

2024-05-28T21:25:24Z

‎Uploading "missing" or N/A values

Nils Duepont: /* Uploading "missing" or N/A values */

2024-05-28T21:24:40Z

‎Uploading "missing" or N/A values

Nils Duepont: /* Uploading "missing" or N/A values */

2023-11-23T11:48:07Z

‎Uploading "missing" or N/A values

Nils Duepont: /* Circumventing typical issues */

2023-11-23T11:44:20Z

‎Circumventing typical issues

Nils Duepont: /* Circumventing typical issues */

2023-11-23T11:44:00Z

‎Circumventing typical issues

Nils Duepont: /* Circumventing typical issues */

2023-11-23T11:42:44Z

‎Circumventing typical issues

Nils Duepont: /* Uploading "missing" or N/A values */

2023-11-23T11:41:49Z

‎Uploading "missing" or N/A values

Nils Duepont: /* Uploading "missing" or N/A values */

2023-11-23T11:41:36Z

‎Uploading "missing" or N/A values

@@ Line 84: / Line 84: @@
 </ol>
-The ''default'' (i.e., valid values) for the column <code>user_na_values</code> is blank. Also notice that only an exclamation mark is allowed for now, and using any other symbol(s) would throw a validation error.
+The ''default'' (i.e., valid values) for the column <code>user_na_values</code> is blank. Also notice that only an exclamation mark is allowed for now, and using any other symbol(s) would throw a validation error. Likewise, make sure that the values are of the same type as the indicator's valid ones (i.e., numeric, string, or [[Coding_rules#Technical_rules|dates]]).
 {| class="wikitable"

@@ Line 71: / Line 71: @@
 </ol>
-The first type usally appears when combining/merging data from different sources on a key (e.g., country-year) and one variable simply does not have data for the focal unit of observation. Typically, you would exclude such observations by omitting them from any analysis (e.g., descriptive statistics, regression models etc.) using only available (or row-wise complete) data. Prime examples for the second type of "missing values" come from quantitative, standardized survey research where the questionnaire includes pre-defined codes that are different from the actual measurement scale (e.g., "No answer", "Refused to answer", "Other" etc.). For such values, the user defines what they "mean" while still excluding them from analyses. Flagging data values with an exclamation mark similarly tells WeSIS to exclude the value from estimations (but still show it in any data table or download).
+The first type usally appears when combining/merging data from different sources on a key (e.g., country-year) and one variable simply does not have data for the focal unit of observation. Typically, you would exclude such observations by omitting them from any analysis (e.g., descriptive statistics, regression models etc.) using only available (or row-wise complete) data. Prime examples for the second type of "missing values" come from quantitative, standardized survey research where the questionnaire includes pre-defined codes that are different from the actual measurement scale (e.g., "No answer", "Refused to answer", "Other" etc.). For such values, the user defines what they "mean" while still excluding them from analyses. Flagging data values with an exclamation mark similarly tells WeSIS to exclude these values from estimations (but still show it in any data table or download). This way of flagging missing data is very generic to account for the peculiarity of the CRC's projects, and give the data providers room for defining "missings" in line with their intention. Still, a certain set of [[Missing values|"missing codes"]] has been agreed upon and it is ''strongly recommended'' to comply whenever you see fit with your indicator to ensure consistency across WeSIS.
-It is recommended to make use of the new way, although WeSIS still accepts data formatted the old way as there may be instances where it makes sense to combine both (for example, using <code>N/A</code> for data when an IO was not yet founded, but a <code>9</code> for coding "Unclear membership status").
+It is recommended to make use of the new way, although WeSIS still accepts data formatted the old way as there may be instances where it makes sense to combine both (for example, using <code>N/A</code> for data when an IO was not yet founded, and a <code>9</code> for coding "Unclear membership status"; see also the recommended set of [[Missing values|"missing codes"]]).

← Older revision		Revision as of 21:25, 28 May 2024
Line 73:		Line 73:
	The first type usally appears when combining/merging data from different sources on a key (e.g., country-year) and one variable simply does not have data for the focal unit of observation. Typically, you would exclude such observations by omitting them from any analysis (e.g., descriptive statistics, regression models etc.) using only available (or row-wise complete) data. Prime examples for the second type of "missing values" come from quantitative, standardized survey research where the questionnaire includes pre-defined codes that are different from the actual measurement scale (e.g., "No answer", "Refused to answer", "Other" etc.). For such values, the user defines what they "mean" while still excluding them from analyses. Flagging data values with an exclamation mark similarly tells WeSIS to exclude the value from estimations (but still show it in any data table or download).		The first type usally appears when combining/merging data from different sources on a key (e.g., country-year) and one variable simply does not have data for the focal unit of observation. Typically, you would exclude such observations by omitting them from any analysis (e.g., descriptive statistics, regression models etc.) using only available (or row-wise complete) data. Prime examples for the second type of "missing values" come from quantitative, standardized survey research where the questionnaire includes pre-defined codes that are different from the actual measurement scale (e.g., "No answer", "Refused to answer", "Other" etc.). For such values, the user defines what they "mean" while still excluding them from analyses. Flagging data values with an exclamation mark similarly tells WeSIS to exclude the value from estimations (but still show it in any data table or download).

−	It is recommended to make use of the new way, although WeSIS still accepts data formatted the old way as there may be instances where it makes sense to combine both (for example, using <code>N/A</code> for data when an IO was not yet founded, but a <code>9</code> for coding ~~say~~ "Unclear membership status").	+	It is recommended to make use of the new way, although WeSIS still accepts data formatted the old way as there may be instances where it makes sense to combine both (for example, using <code>N/A</code> for data when an IO was not yet founded, but a <code>9</code> for coding "Unclear membership status").

← Older revision		Revision as of 21:24, 28 May 2024
Line 73:		Line 73:
	The first type usally appears when combining/merging data from different sources on a key (e.g., country-year) and one variable simply does not have data for the focal unit of observation. Typically, you would exclude such observations by omitting them from any analysis (e.g., descriptive statistics, regression models etc.) using only available (or row-wise complete) data. Prime examples for the second type of "missing values" come from quantitative, standardized survey research where the questionnaire includes pre-defined codes that are different from the actual measurement scale (e.g., "No answer", "Refused to answer", "Other" etc.). For such values, the user defines what they "mean" while still excluding them from analyses. Flagging data values with an exclamation mark similarly tells WeSIS to exclude the value from estimations (but still show it in any data table or download).		The first type usally appears when combining/merging data from different sources on a key (e.g., country-year) and one variable simply does not have data for the focal unit of observation. Typically, you would exclude such observations by omitting them from any analysis (e.g., descriptive statistics, regression models etc.) using only available (or row-wise complete) data. Prime examples for the second type of "missing values" come from quantitative, standardized survey research where the questionnaire includes pre-defined codes that are different from the actual measurement scale (e.g., "No answer", "Refused to answer", "Other" etc.). For such values, the user defines what they "mean" while still excluding them from analyses. Flagging data values with an exclamation mark similarly tells WeSIS to exclude the value from estimations (but still show it in any data table or download).

−	It is recommended to make use of the new way, although WeSIS still accepts data ~~formated~~ the old way as there may be instances where it makes sense to combine both (for example, using <code>N/A</code> for data when an IO was not yet founded, but a <code>9</code> for coding say "Unclear membership status").	+	It is recommended to make use of the new way, although WeSIS still accepts data formatted the old way as there may be instances where it makes sense to combine both (for example, using <code>N/A</code> for data when an IO was not yet founded, but a <code>9</code> for coding say "Unclear membership status").

@@ Line 71: / Line 71: @@
 </ol>
-The first type usally appears when combining/merging data from different sources on a key (e.g., country-year) and one variable simply does not have data for the focal unit of observation. Typically, you would exclude such observations by omitting them from any analysis (e.g., descriptive statistics, regression models etc.) using only available (or row-wise complete) data. Prime examples for the second type of "missing values" come from quantitative, standardized survey research where the questionnaire includes pre-defined codes that are different from the actual measurement scale (e.g., "No answer", "Refused to answer", "Other" etc.). For such values, the user defines what they "mean" while still excluding them from analyses.
+The first type usally appears when combining/merging data from different sources on a key (e.g., country-year) and one variable simply does not have data for the focal unit of observation. Typically, you would exclude such observations by omitting them from any analysis (e.g., descriptive statistics, regression models etc.) using only available (or row-wise complete) data. Prime examples for the second type of "missing values" come from quantitative, standardized survey research where the questionnaire includes pre-defined codes that are different from the actual measurement scale (e.g., "No answer", "Refused to answer", "Other" etc.). For such values, the user defines what they "mean" while still excluding them from analyses. Flagging data values with an exclamation mark similarly tells WeSIS to exclude the value from estimations (but still show it in any data table or download).
 It is recommended to make use of the new way, although WeSIS still accepts data formated the old way as there may be instances where it makes sense to combine both (for example, using <code>N/A</code> for data when an IO was not yet founded, but a <code>9</code> for coding say "Unclear membership status").

@@ Line 177: / Line 177: @@
 ''Issue: What about entities that were part of others in general? Or about "Yugoslavia" in particular?
-: Answer: COW has unique codes and defined relations for its entities. Thus, COW allows for -- and WeSIS accepts data -- for the entire time period, but you have to ensure that the data is attributable to the focal entity. Some fuzziness is unavoidable, e.g. does the data point for 345 in 1968 refer to "core" Serbia as part of the federation or to "Yugoslavia" in general?
+: Answer: COW has unique codes and defined relations for its entities. Thus, COW allows for – and WeSIS accepts data – for the entire time period, but you have to ensure that the data is attributable to the focal entity. Some fuzziness is unavoidable, e.g. does the data point for 345 in 1968 refer to "core" Serbia as part of the federation or to "Yugoslavia" in general?
 : Solution: Make use of an optional column "comment" in the template and WeSISpedia to document the "proper" assignment. Think about providing two indicators – a "raw" and an "imputed" one – if it is (a) justifiable and (b) methodologically sensible to "impute" or "disaggregate" the data.

@@ Line 170: / Line 170: @@
 ''Issue: WeSIS gives me '''CountryName_mismatch''' error, but I am sure there is an issue with the entity list...''
-: Solution: Get in touch with A01, particularly Nils, to discuss the issue.
+: Solution: Get in touch with the INF project, particularly Nils Düpont, to discuss the issue.
 ''WeSIS says there is a mismatch between a country name and the COW code...''

← Older revision		Revision as of 11:41, 23 November 2023
Line 101:		Line 101:


−	'''Uploading N/A values (old way)'''	+	'''Uploading N/A values (the old way)'''

	To upload an N/A value the user should:		To upload an N/A value the user should: