The request
{
"name": "wever_dataset-quality-check",
"arguments": {
"dataset": {
"name": "Orders export"
},
"claims": {
"max_missing_percent": 5,
"max_duplicate_percent": 0,
"max_invalid_percent": 0
},
"fields": [
{
"name": "order_id",
"type": "string",
"required": true
},
{
"name": "amount",
"type": "number",
"min": 0
}
],
"sample": [
{
"id": "r1",
"values": [
{
"field": "order_id",
"value": "A-1"
},
{
"field": "amount",
"value": "19.99"
}
]
},
{
"id": "r2",
"values": [
{
"field": "order_id",
"value": "A-2"
},
{
"field": "amount",
"value": "5.00"
}
]
},
{
"id": "r3",
"values": [
{
"field": "order_id",
"value": "A-2"
},
{
"field": "amount",
"value": "5.00"
}
]
},
{
"id": "r4",
"values": [
{
"field": "order_id",
"value": "A-4"
},
{
"field": "amount",
"value": "-3"
}
]
},
{
"id": "r5",
"values": [
{
"field": "order_id",
"value": "A-5"
},
{
"field": "amount",
"value": "12.50"
}
]
}
]
}
}What the report returns
Verdict: dataset_quality_deficient.
| Finding, duplicates above claim | 20% of records are duplicates, above the 0% claimed: r2 = r3. |
|---|
| Finding, invalid above claim | 1 field carries invalid values more often than the 0% claimed: amount at 20%. |
|---|
| Fix, duplicates above claim | Remove the duplicate records, or state the duplicate share the data carries. |
|---|
| Fix, invalid above claim | Correct or drop the invalid values, or state the invalid share the data carries. |
|---|
Orders export falls short of its quality claims: duplicates above claim and invalid above claim.
An orders export claims no duplicates, no invalid values and at most 5% empty values. Its sample of five records holds every order id and amount, so the empty share sits inside the claim. Records r2 and r3 repeat each other, and r4 carries an amount below the field's minimum of zero, so 20% of records are duplicates and 20% of amounts are invalid. The report states that removing the duplicate and correcting the amount, or stating the shares the data carries, moves the verdict to dataset_quality_sound.