For AI agents
Most people writing against this library now have an agent open beside them. The rest of these docs are written for a human reading one page at a time, which is the wrong shape for that: an agent needs the whole surface at once, in plain text, with the exact spellings.
So there is one file, in thirty-five sections. It carries every public type and member of the four packages, the behaviour behind them, the JSON a client sends and receives, every error string, and the traps that produce code which compiles and is quietly wrong. An agent that reads it needs no other page here.
How to use it
Either hand it over, or let the agent fetch it.
Read https://doc.dynamicwhere.com/llms.txt before writing any
DynamicWhere.ex code. It is the complete API surface.That works with any agent that can read a URL. If yours cannot, use the copy button below and paste the file into your context.
Why plain text rather than a nicer page
What an agent consumes is the text. Syntax highlighting, cards and collapsible sections cost context and carry no meaning once the markup is stripped.
llms.txt is also the path agents and crawlers already look for, so pointing at it needs no explanation.What is in it
- Shapes and results. Every property of
Condition,ConditionGroup,ConditionSet,OrderBy,GroupBy,AggregateBy,PageBy,Filter,SegmentandSummarywith its type and default, and what each member of the three result types holds. - Enums, verbatim, with their numbers — the numbers a JSON body must send when the host registers no string enum converter — including the case-insensitive
Ivariants and the one-mspelling ofSumation. - All twenty-eight extension methods with their real signatures, what each one validates, and which have no synchronous or in-memory form. Plus the generated predicate for every operator, value coercion per
DataType, and how field paths resolve. - The JSON on the wire. Which body binds to which shape, casing and enum converters, how
valuesmust be typed, the result envelope, what rows look like per method, and copy-paste recipes. - Validation and errors. Every rule in the order it is checked, all thirty error strings with what raises them, and the other exception types a caller can receive.
- The policy layer. All twenty-two attributes with their parameters, the six precedence levels, enforcement tier by tier, the transform chain with the exact output of every mask and generalize mode, the group floor, dynamic rules and stores, the admin API, and all twenty-two policy error codes.
- The reflection cache. Every
CacheExposemember, the options and their ranges, the presets, and what eviction actually does. - Fifty-four traps that produce silently wrong code: sixteen for the query engine, thirty-eight for policies. A mask without
[DwNoOrder]leaking through sorting is the one an agent reproduces most often, because the attribute reads as sufficient on its own. - Worked examples for a filter, a summary, a segment, an endpoint, a fully protected entity and the policy wiring around it.
The file
This is the exact content served at /llms.txt. The page reads it at build time, so the two are never out of step.
llms.txt · 5,272 lines
# DynamicWhere.ex — complete reference for coding agents
> A .NET library that turns JSON filter objects into Entity Framework Core LINQ queries, with an
> opt-in field-level policy layer that decides what each caller may filter, sort, select, group,
> aggregate and see, and a reflection cache that needs no setup.
>
> Version 3.2.0 · targets net6.0 · runs on .NET 6, 7, 8, 9, 10 · EF Core 6+ · MIT
> Docs: https://doc.dynamicwhere.com · Source: https://github.com/Sajadh92/DynamicWhere.ex
This file is the whole library in one pass: every public type and member of the four packages, the
behaviour behind them, the JSON a client sends and receives, every error, and the traps that produce
code which compiles and is quietly wrong. It was written from the source and checked by running the
library. No other page is needed; where another page disagrees with this file, this file is right.
Where a name is not in this file, it does not exist — do not invent members.
```
dotnet add package DynamicWhere.ex --version 3.2.0
dotnet add package DynamicWhere.ex.Policies.Redis # optional, same version as the core
dotnet add package DynamicWhere.ex.Policies.EntityFrameworkCore # optional, same version as the core
dotnet add package DynamicWhere.ex.Policies.AspNetCore # optional, same version as the core
```
How to read it:
- Querying with JSON filters: sections 1–11. Read 11 (traps) before writing code.
- Field-level policies: 12 and 13 first, then 14–31 as needed. Read 32 (traps) before writing policy attributes.
- Errors: 8 (query engine) and 30 (policies).
- The reflection cache needs nothing by default: 33.
```
Contents
1 Packages, dependencies, namespaces
2 Shapes and results
3 Enums — `DynamicWhere.ex.Enums`
4 Conditions: operators, values, generated predicates
5 Field paths
6 Extension methods
7 Grouping and aggregation — Group<T>, Summary<T>, ToList(Summary)
8 Validation and core error strings
9 JSON wire format
10 JSON recipes
11 Traps — query engine
12 Policy lifecycle and configuration
13 Guarded queries
14 Policy attributes
15 Policy enums
16 Precedence
17 Enforcement by tier and dry run
18 Transforms
19 Group floor and transformed summaries
20 Model validation
21 Trace, explain and custom providers
22 Audit
23 Discovery: catalogue, schema, simulation
24 Token vaults
25 Dynamic rules
26 Rule stores and StorePolicyProvider
27 Redis package — DynamicWhere.ex.Policies.Redis
28 Entity Framework Core package — DynamicWhere.ex.Policies.EntityFrameworkCore
29 ASP.NET Core package — DynamicWhere.ex.Policies.AspNetCore
30 Policy error codes
31 Policy recipes and inference channels
32 Traps — policies
33 Reflection cache
34 Version history, breaking changes and limits
35 Worked examples (C#)
```
## 1. Packages, dependencies, namespaces
Four packages, always the same version. All target `net6.0` and run on .NET 6–10.
```
DynamicWhere.ex the query engine, policies, token vault interface, cache
Microsoft.EntityFrameworkCore 6.0.22
System.Linq.Dynamic.Core 1.6.7
Microsoft.Extensions.Configuration.Abstractions 6.0.0
Microsoft.Extensions.Configuration.Binder 6.0.0
Microsoft.Extensions.DependencyInjection.Abstractions 6.0.0
DynamicWhere.ex.Policies.Redis RedisPolicyStore, RedisTokenVault
StackExchange.Redis 2.8.24
DynamicWhere.ex.Policies.EntityFrameworkCore EfPolicyStore, EfTokenVault, DwPolicyDbContext + configurations
Microsoft.EntityFrameworkCore.Relational 6.0.22
DynamicWhere.ex.Policies.AspNetCore MapDwPolicyAdmin, claims adapter, audit middleware
FrameworkReference Microsoft.AspNetCore.App
```
- The engine parses every expression it builds with its own `ParsingConfig`: System.Linq.Dynamic.Core's defaults
with `AreContextKeywordsEnabled = false`. It does not read `ParsingConfig.Default` (since 3.1.0), so a change a host
makes there does not reach DynamicWhere queries.
Namespace of every public type:
```
DynamicWhere.ex.Source Extension (all core extension methods), DwDates, DwDateOptions (date formats, 3.1.0)
DynamicWhere.ex.Classes.Core Condition ConditionGroup ConditionSet OrderBy GroupBy AggregateBy PageBy
DynamicWhere.ex.Classes.Complex Filter Segment Summary
DynamicWhere.ex.Classes.Result FilterResult<T> SegmentResult<T> SummaryResult
DynamicWhere.ex.Enums DataType Operator Connector Direction Intersection Aggregator
DynamicWhere.ex.Exceptions LogicException PolicyException
DynamicWhere.ex.Policies.Source PolicyExtensions (ApplyPolicy) PolicyQueryable<T>
DynamicWhere.ex.Policies.Config DwPolicy DwPolicyOptions DwCaps DwPolicyConfiguration (AddDwPolicies, Bind)
DynamicWhere.ex.Policies.Context DwPolicyContext DwSubject
DynamicWhere.ex.Policies.Attributes DwPolicyAttribute and the 22 Dw*Attribute types
DynamicWhere.ex.Policies.Enums PolicyFeature PolicyEffect PolicyAction PolicyLevel PolicyErrorCode
MaskStrategy GeneralizeMode DatePart DwTier DwSubjectKind StoreFailureMode
DynamicWhere.ex.Policies.Masking IValueTransformer DwTransformContext
DynamicWhere.ex.Policies.Tokens IDwTokenVault InMemoryTokenVault DwToken
DynamicWhere.ex.Policies.Audit IDwAuditSink DwAuditEvent
DynamicWhere.ex.Policies.Discovery DwEntityCatalog PolicySchemaBuilder PolicySchema PolicySchemaField
PolicySchemaNode PolicySchemaRequest PolicySimulator PolicySimulation<TClause>
DynamicWhere.ex.Policies.Resolution IDwPolicyProvider AttributePolicyProvider StorePolicyProvider PolicyResolver
DynamicWhere.ex.Policies.Storage IDwPolicyStore IDwPolicyWritableStore IDwPolicyRefresher InMemoryPolicyStore
PolicyRule PolicyRuleDocument PolicyPayload RuleDetail SealedFields
StoreSnapshot NarrowZone
DynamicWhere.ex.Policies.DTOs PolicyTrace PolicyDecision PolicyExplanation FeatureExplanation FieldPolicy
FieldFacts ForcedPredicate PolicyFragment PolicySource TypePolicy
ValueTransform TransformStage TransformKind MutateStage GeneralizeStage
FormatStage MaskStage TruncateStage DefaultStage
DynamicWhere.ex.Policies.Validation PolicyModelValidator PolicyModelReport
DynamicWhere.ex.Optimization.Cache.Source CacheExpose
DynamicWhere.ex.Optimization.Cache.Config CacheOptions
DynamicWhere.ex.Optimization.Cache.Enums CacheEvictionStrategy CacheMemoryType
DynamicWhere.ex.Optimization.Cache.DTOs CacheStatistics CacheMemoryUsage CacheConfiguration
CachePerformanceEvaluation CacheMonitoringSession
DynamicWhere.ex.Optimization.Cache.Input AccessTrackingInput<TKey> CacheFullCheckInput HealthAlertsInput MemoryCalculationInput
DynamicWhere.ex.Optimization.Cache.Output CacheCounts CacheDatabases TrackingCounts
DynamicWhere.ex.Policies.Redis (Redis package) RedisPolicyStore RedisTokenVault
DynamicWhere.ex.Policies.EntityFrameworkCore (EF Core package) DwPolicyDbContext EfPolicyStore EfTokenVault
DwPolicyRuleConfiguration DwPolicyTokenConfiguration
DwPolicyVersionConfiguration DwPolicyRuleRecord DwPolicyTokenRecord
DwPolicyVersionRecord
DynamicWhere.ex.Policies.AspNetCore (ASP.NET Core package) DwPolicyEndpoints (MapDwPolicyAdmin)
DwPolicyAdminOptions DwClaimsOptions DwClaimsAdapter
ClaimsPrincipalPolicyExtensions DwPolicyHttpContextExtensions
DwPolicyAuditMiddleware DwPolicyAuditMiddlewareExtensions
SchemaRequest ExplainRequest SimulateRequest RuleRequest
```
The typical usings: `DynamicWhere.ex.Source`, `.Classes.Core`, `.Classes.Complex`, `.Classes.Result`, `.Enums`;
add `.Policies.Source` for `ApplyPolicy`, `.Policies.Config` for `AddDwPolicies`/`DwPolicy`,
`.Policies.Context` for `DwPolicyContext`, `.Policies.Attributes` and `.Policies.Enums` on entities.
---
## 2. Shapes and results
Three request shapes. Each is a plain class with public get/set properties and no JSON attributes.
- `Filter` — where → order → page → select, one query; returns `FilterResult<T>`
- `Summary` — where → group + aggregate → having → order → page, one query; returns `SummaryResult`
- `Segment` — several condition sets combined with Union / Intersect / Except into one query, then ordered, paged
and projected like a filter; returns `SegmentResult<T>`
Unset enum properties take member 0. Lists start empty unless the property type has `?`.
```
Core — DynamicWhere.ex.Classes.Core
Class Property Type Default Meaning
Condition Sort int 0 order in its group; unique among that group's Conditions
Field string? null member path on T; required
DataType DataType Text predicate form and value parsing (section 4)
Operator Operator Equal
Values List<object> [] operands; count fixed by Operator; null is read as []
ConditionGroup Sort int 0 order among sibling sub-groups; unique among them
Connector Connector And joins every child of this group
Conditions List<Condition> []
SubConditionGroups List<ConditionGroup> [] nests to any depth
ConditionSet Sort int 0 order of the set operations; unique in the Segment
Intersection Intersection? null required on every set but the lowest Sort (ignored there)
ConditionGroup ConditionGroup new() this set's where
OrderBy Sort int 0 lower applies first; duplicates allowed, ties keep list order
Field string? null member path on T; required
Direction Direction Ascending
PageBy PageNumber int 0 1-based; must be >= 1
PageSize int 0 must be >= 1
GroupBy Fields List<string> [] >= 1 paths, unique (case-insensitive), each ending on a simple type
AggregateBy List<AggregateBy> [] optional
AggregateBy Field string? null member path on T; may be omitted only for Count
Alias string? null required identifier; names the result column
Aggregator Aggregator Count
```
```
Complex — DynamicWhere.ex.Classes.Complex
Filter ConditionGroup ConditionGroup? null null = no where
Selects List<string>? null null = whole entities; [] throws MustHasFields
Orders List<OrderBy>? null [] = no ordering; guarded: T's DefaultOrder (section 13)
Page PageBy? null null = every row
Segment ConditionSets List<ConditionSet> [] [] = runs as ToListAsync(new Filter { Selects, Orders, Page })
Selects List<string>? null projected last, after ordering and paging
Orders List<OrderBy>? [] applied in SQL to the combined rows; [] as for Filter
Page PageBy? null applied in SQL after ordering
Summary ConditionGroup ConditionGroup? null where, before grouping
GroupBy GroupBy? null required; null throws ArgumentNullException
Having ConditionGroup? null each Condition.Field is an AggregateBy.Alias, not a path
Orders List<OrderBy>? null each Field is a GroupBy field or an Alias
Page PageBy? null pages the groups
```
```
Results — DynamicWhere.ex.Classes.Result
FilterResult<T> PageNumber:int PageSize:int PageCount:int TotalCount:int Data:List<T> QueryString:string? Policy:PolicyTrace?
SegmentResult<T> : FilterResult<T>, no members of its own
SummaryResult a separate class, not a FilterResult: the same seven members with Data:List<dynamic>
FilterResult<dynamic> is what ToListDynamic and ToListAsyncDynamic return
PageNumber Page.PageNumber; 0 when Page is null
PageSize Page.PageSize; 0 when Page is null
TotalCount counted before paging: rows matching the where (Filter), groups left after Having (Summary),
rows left after the set operations (Segment)
PageCount (int)Math.Ceiling((double)TotalCount / PageSize), or 1 when no Page was sent (0 with no rows),
on FilterResult, SummaryResult and SegmentResult alike. Before 3.1.0 an unpaged Filter or Summary
reported PageCount = TotalCount (one page per row) and an unpaged Segment with sets reported 0.
Unpaged, PageNumber and PageSize stay 0 — except on a guarded query when DwCaps.DefaultPageSize is
set, which gives it page 1 at min(DefaultPageSize, MaxPageSize) (section 12)
Data the page. Typed rows are whole T objects even with Selects: unselected members keep constructor defaults
QueryString with getQueryString: true, EF Core ToQueryString() of the data query (after order, page and projection;
never the count query), otherwise null. Always null for Segment, which has no such parameter.
On a non-EF source it holds the text "The given 'IQueryable' does not support generation of query strings."
Under ApplyPolicy in the Strict tier, getQueryString: true throws QueryStringDenied
Policy PolicyTrace written by the ApplyPolicy terminals (section 21); null otherwise. Under the Strict
tier null too, unless DwPolicyOptions.IncludeTraceInResult is true (3.1.0); LastTrace still holds it
```
---
## 3. Enums — `DynamicWhere.ex.Enums`
Values are implicit, in declaration order. None is `[Flags]`. Numbers are what a JSON body sends when the
host has no string enum converter (section 9).
```
DataType Text=0 Guid=1 Number=2 Boolean=3 DateTime=4 Date=5 Enum=6
Operator Equal=0 IEqual=1 NotEqual=2 INotEqual=3
Contains=4 IContains=5 NotContains=6 INotContains=7
StartsWith=8 IStartsWith=9 NotStartsWith=10 INotStartsWith=11
EndsWith=12 IEndsWith=13 NotEndsWith=14 INotEndsWith=15
In=16 IIn=17 NotIn=18 INotIn=19
GreaterThan=20 GreaterThanOrEqual=21 LessThan=22 LessThanOrEqual=23
Between=24 NotBetween=25 IsNull=26 IsNotNull=27
Connector And=0 Or=1
Direction Ascending=0 Descending=1
Intersection Union=0 Intersect=1 Except=2
Aggregator Count=0 CountDistinct=1 Sumation=2 Average=3 Minimum=4 Maximum=5 FirstOrDefault=6 LastOrDefault=7
```
- `Sumation` has one `m`. There are no `Sum`, `Avg`, `Min` or `Max` members.
- The `I` prefix means case-insensitive and exists only for text operators (`IEqual`, `IContains`, `IIn` …).
- `FirstOrDefault` / `LastOrDefault` return the smallest / largest value in the group, not the first / last row.
---
## 4. Conditions: operators, values, generated predicates
### DataType × Operator
A pair not listed throws `LogicException("Unsupported combination of DataType 'Guid' and Operator 'GreaterThan'.")`
when the predicate is built, after the value checks have passed.
```
DataType Operators accepted
Text Equal NotEqual Contains NotContains StartsWith NotStartsWith EndsWith NotEndsWith In NotIn,
the I-variant of each of those ten, IsNull IsNotNull (no ranges)
Guid Equal NotEqual In NotIn IsNull IsNotNull
Number Equal NotEqual GreaterThan GreaterThanOrEqual LessThan LessThanOrEqual Between NotBetween
In NotIn IsNull IsNotNull
Boolean Equal NotEqual IsNull IsNotNull
DateTime Equal NotEqual GreaterThan GreaterThanOrEqual LessThan LessThanOrEqual Between NotBetween
IsNull IsNotNull (no In / NotIn)
Date same as DateTime
Enum Equal NotEqual In NotIn IsNull IsNotNull
Contains NotContains StartsWith NotStartsWith EndsWith NotEndsWith (no I-variants)
```
Pick the DataType from the member's CLR type. It is never checked against the member, so a mismatch is not
refused: `Guid` on a `string` member matches nothing, `Text` on a numeric member is converted by the parser.
```
Member type DataType Notes
string Text also Enum, for a string column holding enum names
int long short byte decimal double … Number also works on an enum-typed member with its numeric value
bool / bool? Boolean
Guid / Guid? Guid
DateTime DateTime exact instant comparison
DateTime Date compares .Date on both sides
DateTime? / DateTimeOffset / DateTimeOffset? / DateOnly / DateOnly?
DateTime also Date; the predicate is built from the member's type
(3.1.0; before it, DataType.Date on DateTime? threw)
enum (stored as int or as string) Enum Equal NotEqual In NotIn IsNull IsNotNull; value by member name (any
case) or by number. Contains/StartsWith/EndsWith on an enum-typed
member throw ParseException ("No applicable method 'Contains' exists
in type '<Enum>'") whatever the storage
List<string> and other simple-value — a Where on the collection itself throws ParseException ("Operator '=='
collections incompatible with operand types 'List`1' and 'String'"); ordering by it works
```
### Generated predicate
The predicate is a System.Linq.Dynamic.Core string. `f` is the member access, `V` the value after trimming and escaping.
```
Operator Predicate
Equal / NotEqual f != null && f == V f != null && f != V
Contains / NotContains f != null && f.Contains(V) f != null && !f.Contains(V)
StartsWith / NotStartsWith f != null && f.StartsWith(V) f != null && !f.StartsWith(V)
EndsWith / NotEndsWith f != null && f.EndsWith(V) f != null && !f.EndsWith(V)
I-variants (Text only) f.ToLower() in place of f; V lowered in C# with string.ToLower() (server culture)
In f != null && (f == V1 || f == V2 || ...)
NotIn f != null && (f != V1 && f != V2 && ...)
GreaterThan … LessThanOrEqual f != null && f > V (>=, <, <=)
Between f != null && f >= V1 && f <= V2 inclusive; bounds used in the order given
NotBetween f != null && (f < V1 || f > V2)
IsNull / IsNotNull f == null f != null
V by DataType Text, Guid, Enum "v" Number, Boolean v (unquoted)
DateTime DateTime.Parse("canonical") for a DateTime member;
DateTimeOffset.Parse("canonical") for a DateTimeOffset member
Date the same call with .Date on both sides; f becomes f.Date, or f.Value.Date
where the member is nullable
```
- Every operator except IsNull / IsNotNull starts with `f != null &&`. Negated
operators (NotEqual, NotIn, NotContains, NotBetween …) therefore never return rows whose member is null.
- The date types are the exception: since 3.1.0 they resolve the member's type first and emit the guard only
where the member is actually nullable (section 4, Date / DateTime). Every other DataType still guards
unconditionally.
- IsNull on a non-nullable value member matches nothing; IsNotNull matches everything. On a non-nullable date
member of the entity itself the predicate is now the constant `false` / `true` rather than a comparison with
null. Reached through a navigation, IsNull / IsNotNull test the navigation instead (Date / DateTime below).
- Values are trimmed; `\` and `"` are escaped. A value matches literally and cannot close the literal or inject
predicate text. Values are inlined as literals, not SQL parameters.
- A list of more than 32 values (3.1.0) is nested as a balanced tree of flat chains of at most 32 terms, all joined by
the same operator: 50 values of an `In` become `f != null && ((f == V1 || … || f == V25) || (f == V26 || … || f == V50))`.
This covers `In`, `NotIn`, `IIn` and `INotIn` on Text and `In` / `NotIn` on Guid, Number and Enum. A list of 32 or
fewer is written exactly as the table shows, so its predicate and SQL are unchanged, and a longer list returns the
same rows.
- Security fix. Before 3.1.0 every list was one flat chain, one level of nesting per value, and EF Core and the
expression compiler walk that tree recursively: a condition carrying about seven hundred values overflowed the
request thread's stack. A stack overflow ends the process, and no `catch` can stop it. Guarded and unguarded
queries alike, since before 3.0.0.
- Under `ApplyPolicy`, `Caps.MaxConditionValues` (default 1000) also bounds the values of one condition (section 12).
- `Between` with V1 > V2 matches nothing; `NotBetween` with V1 > V2 matches every non-null row.
- Plain text operators add no case handling: in memory they are ordinal; in SQL the provider and collation
decide (SQLite: `==` and Contains case-sensitive, StartsWith/EndsWith case-insensitive for ASCII). The
I-variants are case-insensitive everywhere. `ToLower()` on the column can defeat an index.
### Values
Each element of `Values` is first normalized to a string:
```
Element Normalized to
null, JsonElement Null ""
string itself
bool "true" / "false"
JsonElement String its string
JsonElement Number the raw token as sent ("1.50" stays "1.50")
JsonElement True / False "true" / "false"
JsonElement Array / Object the raw JSON text
DateTime "yyyy-MM-ddTHH:mm:ss.FFFFFFF", no zone marker (3.1.0; was "MM/dd/yyyy HH:mm:ss");
"yyyy-MM-ddTHH:mm:ss.FFFFFFFzzz" for a Kind Local value compared under DataType.DateTime
with a DateTimeOffset member (3.1.0; see Date / DateTime below)
DateTimeOffset "yyyy-MM-ddTHH:mm:ss.FFFFFFFzzz" (3.1.0)
DateOnly "yyyy-MM-dd" (3.1.0)
other IFormattable ToString(null, InvariantCulture): 12.5 -> "12.5", Guid -> "D" form, enum -> member name
anything else ToString()
```
Then checked per DataType. A failed check throws `InvalidFormat` — or, for a date, `AmbiguousDateFormat`.
```
DataType Check Send
Text none a string; a null element is "" and matches empty strings only
Enum none member name in any case ("Pending", "pending") or its number
Guid Guid.TryParse any Guid format: "D", upper-case, "N" (no hyphens)
Number TryParse as byte/short/int/long/float/double/decimal a JSON number or numeric string: 12, -3.5, "15.5", "1e3"
(server culture)
Boolean bool.TryParse true / false as JSON booleans or strings in any case; 1 and 0 fail
DateTime ISO 8601 / year-first / a declared format (3.1.0) ISO 8601: "2024-06-15T14:30:00", or with Z / an offset
Date ISO 8601 / year-first / a declared format (3.1.0) ISO 8601 date: "2024-06-15"; the time is dropped on both sides
```
- **Null:** a null element is `""`. Text and Enum compare with the empty string; every other DataType throws
`InvalidFormat`. To test for NULL use `IsNull` / `IsNotNull` with `"values": []`.
- **Number:** the token is embedded unquoted exactly as sent. A thousands separator (`"1,000"`) or `"NaN"` /
`"Infinity"` passes the culture TryParse, then throws `System.Linq.Dynamic.Core.Exceptions.ParseException`
when the query is built. Send invariant literals without separators.
- **Date / DateTime (3.1.0 rewrote this):** the predicate is built from the member's own type.
- **Which texts are dates (3.1.0).** Read against explicit formats, never the lenient parser; the server's culture
and calendar decide nothing.
- Always accepted: ISO 8601 extended calendar dates — `"2026-09-01"` (also `"2026-9-1"`), optionally `T` or a
space and a time (`"12:30"`, `"12:30:15"`, `"12:30:15.123"`), optionally `Z` or an offset (`"+03:00"`,
`"+0300"`, `"+03"`) — and year-first dates `"2026/09/01"`, `"2026.09.01"` with the same optional time. A
lowercase `t` or `z` and a comma before the fraction are accepted, and a fraction longer than seven digits (Go
and Java write nine) is cut to seven, the 100 ns a `DateTime` holds.
- The other ISO 8601 forms are `InvalidFormat`: basic (`"20260901"`), week (`"2026-W36-2"`), ordinal
(`"2026-244"`) and reduced precision (`"2026-09"`, `"2026-09-01T12"`).
- A numeric date that leads with a day or a month — `"01/09/2026"`, `"15/09/2026"`, `"09/15/2026"`,
`"01.09.2026"`, `"01-09-2026"`, `"1/9/26"` — throws `AmbiguousDateFormat` whatever its numbers, with the field
as `LogicException.Subject` (under `ApplyPolicy`, the name the caller wrote, so an alias is not undone). By
shape, not value: refusing only the values with two readings would fail on the 5th of the month and pass on
the 15th.
- Anything else is `InvalidFormat`, including `"12:00"`, `"1/9"`, `"Sep 2026"`, `"1 September 2026"` — which
the lenient parser used to accept as today at noon, a day of the current year (9 January or 1 September, by
the host's culture), and 1 September.
- A deployment declares a local form once at startup, and it is read with the invariant culture alongside ISO:
`DwDates.Configure(o => o.Formats.Add("dd/MM/yyyy"))` makes `"01/09/2026"` 1 September everywhere. See
"Date formats" below.
- Validation and the builder read a value with the same reader and the member's own type, so a value that passes
validation always builds.
- A `DateTimeOffset` member is compared against a `DateTimeOffset` literal, normalised to UTC; a value carrying
no zone is read as UTC, so `DataType.Date` names the day the caller wrote. On Npgsql `DataType.Date` becomes
`date_trunc('day', col AT TIME ZONE 'UTC')`.
- The member's day under `DataType.Date` is the provider's: its UTC day on PostgreSQL, where `timestamptz` keeps
no offset, but the day in its own offset in memory (and on a provider that stores the offset, such as SQL
Server `datetimeoffset`). A row at `2026-09-01T01:00+03:00` is 31 August on PostgreSQL and 1 September in
memory. The value's day is always its UTC day, so send a date with no zone for a day comparison.
- A `DateTime` member is compared against a `DateTime` literal and keeps the older time-zone behaviour: a value
with `Z` or an offset converts to server local time first (on a +03:00 server `"2024-01-01T10:00:00Z"`
compares as 13:00). Send it in the convention the column stores.
- A nullable member is unwrapped under its guard (`f.Value`, `f.Value.Date`). A non-nullable member on the entity
itself gets no guard, and `IsNull` / `IsNotNull` answer `false` / `true` — `WHERE FALSE` and no predicate on
Npgsql, for `DateTime` and `DateTimeOffset` alike. Reached through a navigation (`Approval.ApprovedAt`), each
navigation is guarded instead (`Approval != null && …`), and `IsNull` / `IsNotNull` test the navigation: a
provider reads the member of a missing approval as NULL, and 3.0.0 answered by it the same way.
- `Having` names an alias, so the type comes from the aggregate it stands for: `Minimum`, `Maximum`,
`FirstOrDefault` and `LastOrDefault` have the member's type (nullable if the member is), and the predicate is
built exactly as for that member. On Npgsql: `HAVING max(col) > TIMESTAMPTZ '…'`. Count, Sumation and Average
aliases are never dates and keep the unconditional guard.
- A `DateOnly` member (3.1.0) is compared as a day under both date data types, against a `DateOnly(y, m, d)`
constructor — never `DateOnly.Parse`, which the runtime evaluates in the host's calendar and reads
`"2026-09-01"` as the year 1483 on a Thai server. On Npgsql: `WHERE "Day" = DATE '2026-09-01'`.
- Before 3.1.0 every comparison on a `DateTimeOffset` member threw — `InvalidOperationException` ("The binary
operator NotEqual is not defined for the types 'System.DateTimeOffset' and 'System.Object'") on a
non-nullable one, `ParseException` on a nullable one — `DataType.Date` on any nullable date member threw
`ParseException` ("No property or field 'Date' exists in type 'DateTime?'"), and no comparison on a `DateOnly`
member worked under either date data type (`IsNull` and `IsNotNull` did).
- A C# `DateTime`, `DateTimeOffset` or `DateOnly` placed in `Values` is written year-first (the normalizer table
above) and then read like any other value; before 3.1.0 it took the month-first invariant form
(`MM/dd/yyyy HH:mm:ss`), which is now refused.
- A `DateTime` of `Kind` `Local` (`DateTime.Now`, or what Newtonsoft.Json makes of a string carrying an offset),
compared under `DataType.DateTime` with a `DateTimeOffset` or `DateTimeOffset?` member — a `Having` alias over
such a member's aggregate included — is written with its UTC offset (`2026-09-17T15:00:00+03:00`), so it
filters on the moment it holds. Without the offset the member would read it as UTC, three hours away on a
host at UTC+3, with no error.
- Every other `DateTime` is written with no zone, as before: under `DataType.Date`, so `DateTime.Today` compares
the day it was written for (with its offset, local midnight on the 17th is the 16th in UTC on a host ahead of
UTC); on a `DateTime` member, which holds wall-clock time, and on a `DateOnly` member; and a `DateTime` of
`Kind` `Utc` or `Unspecified`, which a `DateTimeOffset` member reads as UTC.
- Text values, such as JSON strings bound by System.Text.Json, are never rewritten.
- **Enum:** a name that is not a member passes validation and throws `ParseException` when the query is built.
- In C#, `Values` is `List<object>`: `Values = { "Engineering" }` or `new List<object> { 1, 2 }`. A `List<string>`
is not assignable.
### Date formats — `DwDates` (3.1.0)
```csharp
namespace DynamicWhere.ex.Source;
public sealed class DwDateOptions
{
public IList<string> Formats { get; } // .NET exact formats, read with InvariantCulture; read-only once configured
public bool IsFrozen { get; }
}
public static class DwDates
{
public static DwDateOptions Options { get; } // frozen; declares nothing until configured
public static bool IsConfigured { get; }
public static void Configure(DwDateOptions options);
public static void Configure(Action<DwDateOptions> configure);
public static DwDateOptions Bind(this DwDateOptions options, IConfiguration section); // extension
}
```
```csharp
DwDates.Configure(o => o.Formats.Add("dd/MM/yyyy")); // once, at startup
DwDates.Configure(new DwDateOptions().Bind(configuration.GetSection("DynamicWhere:Dates")));
// appsettings.json: { "DynamicWhere": { "Dates": { "Formats": [ "dd/MM/yyyy", "dd/MM/yyyy HH:mm" ] } } }
```
- Declared formats are accepted **in addition to** ISO 8601 and year-first dates, which every deployment accepts.
- `Configure` freezes the options and may be called once; a second call throws `InvalidOperationException`. Every
query reads the formats without a lock.
- Refused at `Configure` with `ArgumentException`:
- a blank or malformed format (`"'dd/MM/yyyy"`, `"q"`);
- a format that cannot read back the text it writes, or reads a part of it back differently — `dd/MM/yyyy hh:mm`,
a 12-hour clock with no `tt`, reads 4 PM as 4 AM;
- a format with no year — `dd/MM`, `HH:mm`, `t` — which the parser would complete from the clock, so the same
value would name a different date depending on when the query ran;
- a format with a day but no month — `dd/mm/yyyy`, where `mm` is minutes;
- two formats that read one text as different dates — `dd/MM/yyyy` beside `MM/dd/yyyy`, or `yyyy-dd-MM` against
ISO;
- a format whose own text ISO 8601 or a year-first date already reads — `yyyy-MM-dd`, `yyyy/M/d`,
`yyyy-MM-dd HH:mm:ss`, `yyyy-MM-dd'T'HH:mm:ss'Z'` (3.1.0). Declaring one can only change what such a value
means: a quoted `'Z'` is a letter, not a zone, so that format reads `12:00` as a wall time where ISO 8601 reads
an instant. On a `DateTime` member the ISO reading converts to the host's local time, so off UTC the two
readings differed and every such value was refused as `AmbiguousDateFormat` — on that host only. The refusal is
the same on every host. Checked last, after the rules above, which name a sharper reason;
- two formats that lead with the day and the month in opposite orders, even in different shapes —
`dd/MM/yyyy HH:mm` beside `MM/dd/yyyy` would make `"01/09/2026 00:00"` 1 September and `"01/09/2026"`
9 January. Declare one day/month order.
- A format with a year but no day, such as `yyyy-MM`, is accepted and reads the 1st. So are `dd/MM/yyyy`,
`dd/MM/yyyy HH:mm` and `dd MMM yyyy`: ISO 8601 reads none of them.
- `Bind` throws `InvalidOperationException`, at startup instead of leaving the defaults in force, for a key
nothing answers to (`ErrorOnUnknownConfiguration`) such as a misspelt `Fromats`, and for a single value where the
list belongs: `"Formats": "dd/MM/yyyy"`, or one environment variable `DynamicWhere__Dates__Formats`. Write
`"Formats": [ "dd/MM/yyyy" ]`, or `DynamicWhere__Dates__Formats__0`. It also throws on frozen options.
- A value still has to match: with `dd/MM/yyyy` declared, `"09/15/2026"` is `AmbiguousDateFormat`.
- There is no per-condition format. A condition's value is read with the process-wide formats.
```
Value count, checked before the format (Condition and Having alike)
IsNull IsNotNull 0 else ConditionWithOperator[IsNull-IsNotNull]MustHasNoValues
Between NotBetween 2 else ConditionWithOperator[Between-NotBetween]MustHasOnlyTwoValues
In IIn NotIn INotIn 1+ else ConditionWithOperator[In-IIn-NotIn-INotIn]MustHasOneOrMoreValues
any other operator 1 else ConditionWithOperator[<Operator>]MustHasOnlyOneValue e.g. ConditionWithOperator[Equal]MustHasOnlyOneValue
```
Each condition is checked in this order: Field, value count, value format, DataType/Operator pair.
### Groups and connectors
```
And children joined with &&
Or children joined with ||
```
- A group emits its `Conditions` in Sort order, then its `SubConditionGroups` in Sort order, each in parentheses:
`(c1 && c2 && (sub1) && (sub2))`. Conditions always precede sub-groups, whatever their Sort values.
- A group has one connector. `A AND (B OR C)` is an And group holding A plus an Or sub-group holding B and C.
- A group with no condition at any depth emits nothing: as the root it filters nothing; as a sub-group it is skipped.
- Sort must be unique among a group's Conditions (`AnyListOfConditionsMustHasUniqueSortValue`) and, separately,
among its SubConditionGroups (`AnyListOfSubConditionsGroupsMustHasUniqueSortValue`). A condition and a sub-group
may share a Sort. The group's own Sort is never checked.
---
## 5. Field paths
`Condition.Field`, `OrderBy.Field`, `GroupBy.Fields`, `AggregateBy.Field` and `Selects` entries are dotted paths
from T. Examples with T = Customer:
```
Path Where predicate generated
"TotalSpent" (TotalSpent != null && TotalSpent > 100)
"ContactInfo.Email" (ContactInfo.Email != null && ContactInfo.Email == "a@b.c")
"RegisteredAt.Year" (RegisteredAt.Year != null && RegisteredAt.Year == 2024) any public property of a CLR type
"Orders.TotalAmount" (Orders.Any(i1 => i1.TotalAmount != null && i1.TotalAmount > 100))
"Orders.ShippingAddress.Country" (Orders.Any(i1 => i1.ShippingAddress.Country != null && i1.ShippingAddress.Country == "USA"))
"Orders.OrderItems.Quantity" (Orders.Any(i1 => i1.OrderItems.Any(i2 => i2.Quantity != null && i2.Quantity > 2)))
```
- **Matching.** Segments are public instance properties, matched case-insensitively and rewritten to the
declared name (`orders.totalamount` → `Orders.TotalAmount`). Fields and serializer names (`[JsonPropertyName]`,
`[Column]`) are not recognized: camelCase works, snake_case does not. Spaces around segments and empty segments
are removed (`"Orders . TotalAmount"`, `"Orders..TotalAmount"`). A path of only dots fails.
- **Names that look reserved.** A member named `Root`, `It` or `Parent`, in any case, is an ordinary segment, and so
is an alias named `root`, `it` or `parent`. Before 3.1.0 System.Linq.Dynamic.Core read such a name as its context
keyword: `Root.Name` and `It.Name` addressed the row's own `Name`, `Parent` threw `ParseException`, and an
`AggregateBy.Alias` named `root`, `it` or `parent` failed in `Having` and `Summary.Orders`. Under `ApplyPolicy`
the gate decided on the path the caller named while the query read the row's own column, so a denied value could
be projected and a forced scope reached through such a navigation filtered the wrong column.
- Expressions are parsed with the context keywords off, so `it`, `root` and `parent` name members like any other
identifier. The parser's predefined type names name members too: `String`, `Boolean`, `Char`, `Byte`, `SByte`,
`Int16`, `Int32`, `Int64`, `UInt16`, `UInt32`, `UInt64`, `Single`, `Double`, `Decimal`, `DateTime`,
`DateTimeOffset`, `TimeSpan`, `Guid`, `Math`, `Convert`, `Uri`, `Object` and `Enum`.
- **Names the library refuses (3.1.0).** A path whose first segment is one of the parser's own functions or
literals — `new`, `iif`, `np`, `isnull`, `is`, `as`, `cast`, `true`, `false`, `null`, whatever the letter case —
throws `LogicException("FieldPath[<path>]StartsWithReservedName")`, with that first segment, trimmed, on
`LogicException.Subject`.
- Raised where a path is validated, so every clause a caller writes answers alike, guarded or not: condition
fields, `Orders`, `Selects`, `GroupBy.Fields`, `AggregateBy.Field`, and the target of a `[DwAlias]` the caller
names.
- Only the first segment. `Owner.New` names the member, because the parser looks for a member after a dot. An
alias named after one of these words still works: only the path it stands for is checked.
- Under `ApplyPolicy`, the Convenience tier and any dry run give that code. The Strict tier, outside a dry run,
answers with the clause's `FieldDeniedFor*` code and `FieldPath` `"*"`, as it answers for every name it cannot
use (section 17).
- A `DefaultOrder` entry naming one refuses nothing: a guarded query drops it, as it drops an entry it cannot
read, and `ValidateModel` reports it as an error (sections 13 and 20).
- The parser used to answer instead, because it reads its own functions and literals before it looks for a
member: `New`, `Iif`, `Np`, `IsNull`, `Is`, `As` and `Cast` raised its `ParseException`, `True` and `False` an
`InvalidOperationException`, and `Null` was read as the null literal, so the query returned no rows and no
error. A typed `Selects` entry naming such a member did work, because a typed projection is built without the
parser; it is refused now too, so one rule covers every clause.
- The remedy for such a column: rename the CLR property and map the column with `[Column("New")]`.
- **Errors.** A missing segment or a null / blank path throws `LogicException("ConditionMustHasValidFieldName")`,
the same string for Condition, OrderBy, GroupBy and AggregateBy paths. A null or blank `Selects` entry throws
`ArgumentNullException` instead. Under `ApplyPolicy` in the Strict tier, outside dry run, a path that matches
nothing is refused like a denied field instead, with a `PolicyException` (3.1.0, section 17).
- **What counts as a collection:** arrays, `List<>`, `IList<>`, `ICollection<>`, `IEnumerable<>`, `HashSet<>`,
`ISet<>`. Anything else (`IReadOnlyList<>`, `IReadOnlyCollection<>`, `Collection<>`, `ObservableCollection<>`, a
class deriving from `List<T>`) is a plain object, so a path continuing past it throws ConditionMustHasValidFieldName.
- **Collections in a where.** The next segment resolves on the element type. Each collection level adds one
`.Any(iN => ...)`, and the whole predicate (null guard and negation included) sits inside the innermost Any.
Negation is per element: NotEqual on `Orders.Status` means "some order has a non-null, different status", not
"no order has it". No `All()` or `!Any()` form exists. Two conditions on the same collection become separate
Any() calls and can be satisfied by different elements.
- **Null navigations.** Only the last member is null-guarded. EF Core translates a null reference navigation to
SQL nulls. In memory (LINQ to Objects) a null navigation earlier in the path throws `NullReferenceException`
in Where, Order and SelectDynamic, so populate navigations in in-memory data.
- **Ordering.** A path through a collection is reduced per collection segment (section 6, Order). Ordering by a
whole reference navigation (`"Category"`) is accepted; EF Core orders by its key, while in memory it throws
`InvalidOperationException("Failed to compare two elements in the array.")`.
- Having fields (aliases) and `Summary.Orders` fields (group fields or aliases) are result columns, not paths.
---
## 6. Extension methods
`public static class Extension`, namespace `DynamicWhere.ex.Source` (the source file is spelled `Extention.cs`;
the class is `Extension`). 28 public methods, all generic with `where T : class`. Every asynchronous one also has
overloads taking a `CancellationToken` (3.2.0).
```
On IQueryable<T> query — composable (validates and builds, executes nothing)
Select<T>(List<string> fields) -> IQueryable<T>
SelectDynamic<T>(List<string> fields) -> IQueryable
Where<T>(Condition condition) -> IQueryable<T>
Where<T>(ConditionGroup group) -> IQueryable<T>
Order<T>(OrderBy order) -> IQueryable<T>
Order<T>(List<OrderBy> orders) -> IQueryable<T>
Page<T>(PageBy page) -> IQueryable<T>
Group<T>(GroupBy groupBy) -> IQueryable
Filter<T>(Filter filter) -> IQueryable<T>
FilterDynamic<T>(Filter filter) -> IQueryable
Summary<T>(Summary summary) -> IQueryable
On IQueryable<T> query — terminal
ToList<T>(Filter filter, bool getQueryString = false) -> FilterResult<T>
ToListAsync<T>(Filter filter, bool getQueryString = false) -> Task<FilterResult<T>>
ToListDynamic<T>(Filter filter, bool getQueryString = false) -> FilterResult<dynamic>
ToListAsyncDynamic<T>(Filter filter, bool getQueryString = false) -> Task<FilterResult<dynamic>>
ToList<T>(Summary summary, bool getQueryString = false) -> SummaryResult
ToListAsync<T>(Summary summary, bool getQueryString = false) -> Task<SummaryResult>
ToListAsync<T>(Segment segment) -> Task<SegmentResult<T>>
On IQueryable<T> query — terminal, cancellable (3.2.0)
ToListAsync<T>(Filter filter, CancellationToken cancellationToken) -> Task<FilterResult<T>>
ToListAsync<T>(Filter filter, bool getQueryString, CancellationToken cancellationToken) -> Task<FilterResult<T>>
ToListAsyncDynamic<T>(Filter filter, CancellationToken cancellationToken) -> Task<FilterResult<dynamic>>
ToListAsyncDynamic<T>(Filter filter, bool getQueryString, CancellationToken cancellationToken) -> Task<FilterResult<dynamic>>
ToListAsync<T>(Summary summary, CancellationToken cancellationToken) -> Task<SummaryResult>
ToListAsync<T>(Summary summary, bool getQueryString, CancellationToken cancellationToken) -> Task<SummaryResult>
ToListAsync<T>(Segment segment, CancellationToken cancellationToken) -> Task<SegmentResult<T>>
On IEnumerable<T> query — terminal, in memory
ToList<T>(Filter filter, bool getQueryString = false) -> FilterResult<T>
ToListDynamic<T>(Filter filter, bool getQueryString = false) -> FilterResult<dynamic>
ToList<T>(Summary summary, bool getQueryString = false) -> SummaryResult
```
These do not exist: a synchronous `ToList<T>(Segment)`, `getQueryString` on Segment, any async or composable
method on `IEnumerable<T>`, names such as `ToListFilter` / `ToListAsyncSegment`, a `new()` constraint, and a
`CancellationToken` parameter on the 3.1 signatures: the token overloads are separate methods, so code compiled
against 3.1 still binds.
### Rules for every method
- The first statement is the `[DwEntity(RequirePolicy = true)]` guard: on such a T, a call outside `ApplyPolicy`
throws `PolicyException` `PolicyRequired` (section 13). Then null arguments throw `ArgumentNullException`, then
the validation rules of section 8 throw `LogicException`.
- Composable methods validate and build when called, not when enumerated, so a bad field throws at the call.
- The async terminals are `async` methods: every exception, validation included, surfaces at `await`.
- Validation rewrites the caller's objects in place: path strings get the declared casing (`"price"` → `"Price"`)
in `Condition.Field`, `OrderBy.Field`, `GroupBy.Fields`, `AggregateBy.Field` and `Selects`; null `Values`,
`Conditions`, `SubConditionGroups`, `ConditionSets`, `Fields` and `AggregateBy` become empty lists; the first
`ConditionSet`'s `Intersection` becomes null. Clone a shape before reusing it if that matters.
### Select<T>
```
products.Select(["Id", "Name", "Category.Name", "OrderItems.Quantity"]) builds
e => new Product {
Id = e.Id, Name = e.Name,
Category = e.CategoryId == null ? new Category() : new Category { Id = …, Name = … },
OrderItems = e.OrderItems.AsQueryable().Select(c => new OrderItem { Id = c.Id, Quantity = c.Quantity }).ToList() }
```
- Errors: `fields` empty → `MustHasFields`; a null or blank entry → `ArgumentNullException`; an unknown path →
`ConditionMustHasValidFieldName`; no public parameterless constructor on T →
`LogicException("SelectTypeMustHaveParameterlessConstructor")` with `Subject = typeof(T).Name` (checked at
run time; the constraint is only `class`). Before 3.1.0 that message was an English sentence carrying the type
name inside it.
- Rows are whole T objects. An unselected member keeps what T's parameterless constructor gives it: initializers
run (`= string.Empty` stays `""`, `= new List<X>()` stays empty), everything else is default. Serialized, every
member appears.
- A non-dotted scalar (`"Name"`) binds the value; a non-dotted navigation (`"Category"`) binds the whole related
entity; a non-dotted collection (`"OrderItems"`) binds the whole collection.
- `"Category.Name"` builds a new `Category` holding `Name`, plus `Id` when the nested type has an `Id`.
- A null reference navigation comes back as a placeholder `new Category()` with constructor defaults, never null.
The null test reads `CategoryId` when T has a property of that name (a default FK value also gives the
placeholder); otherwise it tests `Category == null`.
- `"OrderItems.Quantity"` projects each element into a `List<OrderItem>` (with `Id` when present). An empty
collection gives an empty list. Deeper paths recurse the same way.
- Skipped silently, keeping the constructor default: members without a public setter; collections whose element
type has no parameterless constructor; collection members whose declared type cannot hold a `List<TElement>`
(`HashSet<>`, `ISet<>`, arrays); a dotted path into a string or struct member (`"CreatedAt.Year"`).
- If `"Category"` and `"Category.Name"` are both listed, the dotted projection wins. Duplicate paths collapse.
- **EF Core only for reference navigations:** a dotted reference-navigation path reads `EF.Property<T>`, so on an
in-memory source it throws `InvalidOperationException("The EF.Property<T> method may only be used within Entity
Framework LINQ queries.")`. Scalar and collection paths work in memory.
### SelectDynamic<T>
- Validation and errors as `Select<T>`, without the constructor requirement; `Id` is never added.
- Emits one Dynamic LINQ `new(...)` selector. Each row is an instance of a runtime-generated class deriving from
`System.Linq.Dynamic.Core.DynamicClass`, with real public properties. Read it as `dynamic` (`row.Category.Name`)
or by reflection.
- Property names are the path segments, nested like the path; nothing is flattened: `"Category.Name"` is
`row.Category.Name`, never `row.CategoryName`.
```
fields emitted selector row
"Id", "Name" new(Id, Name) { Id, Name }
"Category" new(Category) { Category: <whole entity or null> }
"Category.Name" new(new(Category.Name as Name) as Category) { Category: { Name } }
"Category.Name", "Category.Id" new(new(Category.Name as Name, np(Category.Id) as Id) as Category) { Category: { Name, Id } }
"OrderItems.Quantity" new(OrderItems.Select(v0 => new(v0.Quantity as Quantity)) as OrderItems) { OrderItems: [ { Quantity } ] }
"Category.Vendors.Id" new(new(Category.Vendors.Select(v0 => new(v0.Id as Id)) as Vendors) as Category)
```
- Nested collections get lambda parameters `v0`, `v1`, … one per level.
- On EF Core a missing reference navigation still produces the nested object with null members
(`"category": { "name": null }`); a non-nullable value type directly under a reference navigation (outside a
collection) is wrapped in `np()` and comes back nullable. A whole-navigation entry (`"Category"`) is null when
the navigation is null. In memory, a dotted path through a null navigation throws `NullReferenceException`.
- If `"Category"` and `"Category.Name"` are both listed, the whole-object entry is dropped.
### Where<T>
- `Where<T>(Condition)` validates the condition and adds one predicate (section 4).
- `Where<T>(ConditionGroup)` adds the group's predicate; an empty group leaves the query unchanged.
### Order<T>
```
Order(OrderBy) one OrderBy("<path> asc|desc")
Order(List<OrderBy>) items sorted by Sort (ties keep list order) into one OrderBy("p1 asc,p2 desc");
the first item is the primary key, the rest are then-by keys
```
- A null, blank or unknown `Field` throws `ConditionMustHasValidFieldName`. An empty list leaves the query unchanged.
- Each call starts a new ordering and replaces an earlier one; it is not a then-by. Put every key in one list.
- A path with no collection is emitted as written (`Category.Name asc`). A path through a collection reduces each
collection segment to one value — `Min` ascending, `Max` descending — so rows sort by their best element in the
requested direction:
```
field dir emitted
Tags.Value asc Tags.Min(Value) asc
Tags.Value desc Tags.Max(Value) desc
OrderItems.Product.Name asc OrderItems.Min(Product.Name) asc
OrderItems.UnitPrice asc OrderItems.Select(UnitPrice).DefaultIfEmpty().Min() asc
Orders.OrderItems.Quantity desc Orders.Select(OrderItems.Select(Quantity).DefaultIfEmpty().Max()).DefaultIfEmpty().Max() desc
Labels (List<string>) asc Labels.Min() asc
```
- An empty collection sorts as null for a reference or nullable element type, and as the type default for a
non-nullable value type (via `DefaultIfEmpty()`, so in-memory sorting does not throw).
- A path ending on a collection of non-simple elements (`"Tags"`, `"Posts.Tags"`) throws
`OrderField[<field>]CannotEndOnCollectionOfComplexElements`; sort by a member inside it (`"Tags.Value"`).
Collections of simple values are allowed.
- Provider limits apply: SQLite, for one, cannot ORDER BY a `decimal` column (EF Core throws `NotSupportedException`).
- `Summary.Orders` does not use any of this (section 7).
### Page<T>
- Emits `Skip((PageNumber - 1) * PageSize).Take(PageSize)`. `PageNumber <= 0` → `PageNumberMustBeGreaterThanZero`;
`PageSize <= 0` → `PageSizeMustBeGreaterThanZero`.
- The core sets no upper bound (the policy layer has `MaxPageSize`). A page past the end is empty. `Page` neither
adds nor requires an ordering; order before paging for stable pages. (The guarded `Page` of section 13 takes the
type's declared `DefaultOrder` on a source nothing has ordered; this one never reads it.)
### Filter<T>, FilterDynamic<T>
```
Filter<T> Where(ConditionGroup) -> Order(Orders) -> Page(Page) -> Select(Selects) -> IQueryable<T>
FilterDynamic<T> Where(ConditionGroup) -> Order(Orders) -> Page(Page) -> SelectDynamic(Selects) -> IQueryable
```
- A step runs only when its member is non-null; `new Filter()` returns the query unchanged.
- `Orders` and `Page` run on T before the projection, so they may name fields that are not selected.
- `Selects = []` throws `MustHasFields`; `Orders = []` means no ordering.
- `FilterDynamic<T>` with `Selects` null returns the `IQueryable<T>` itself, so its rows are T.
### ToList, ToListAsync, ToListDynamic, ToListAsyncDynamic (Filter)
```
ToList, ToListAsync Where -> build Order, Page, Select -> COUNT(where-only query) -> data query
ToListDynamic, ToListAsyncDynamic Where -> COUNT(where-only query) -> build Order, Page, SelectDynamic -> data query
```
- Every call runs two queries: a count of the filtered set (ignoring Page) and the data query.
- In the dynamic pair an invalid `Orders`, `Page` or `Selects` throws after the count has already run.
- `ToListAsync` uses EF Core `CountAsync` / `ToListAsync`. `ToListAsyncDynamic` uses EF Core `CountAsync`, then
EF Core's `ToListAsync` over the query's element type: `T` when `Selects` is null, the projection's generated
class otherwise (3.2.0). It used to read through Dynamic LINQ's `ToDynamicListAsync`, asynchronous as well but
with no token to pass on. Both need an EF Core async provider for the count: on a plain
`list.AsQueryable()` they throw `InvalidOperationException` ("The provider for the source 'IQueryable' doesn't
implement 'IAsyncQueryProvider'…"). Use the synchronous methods in memory.
- The overloads taking a `CancellationToken` (3.2.0) pass it to the count and to the read, so a canceled token stops
whichever is running and the call throws `OperationCanceledException` (EF Core's `TaskCanceledException` derives
from it). The overloads without a token pass `CancellationToken.None`.
- `ToListAsync(filter, default)` does not compile: `default` fits both `bool getQueryString` and
`CancellationToken`. Write `false`, `CancellationToken.None` or a named argument.
- `ToListDynamic` rows are `DynamicClass` objects when `Selects` is set, and T instances when it is null.
### ToListAsync<T>(Segment)
```
validate the sets
combine them in Sort order into one query:
T has a primary key: Where(set1 OR|AND set2 ... AND NOT EXISTS(setN row with the same key))
T has no primary key: set1 UNION|INTERSECT|EXCEPT set2 ... (SQL set operators, whole rows)
then exactly as ToListAsync(Filter): Order(Orders) -> Page(Page) -> Select(Selects); COUNT for TotalCount
```
- The sets combine left to right, `((set1 op2 set2) op3 set3)`, in Sort order, not list order. The database answers
one query: only the requested page is read, plus one COUNT query for `TotalCount`.
- Union and Intersect combine the sets' own conditions with OR and AND. Except removes the rows of its set with
`NOT EXISTS`, matched on T's primary key as EF Core maps it (composite, value-converted and inherited keys
included).
- Rows are matched by key, so a tracking query, `AsNoTracking()` and `Selects` all return the same rows.
- A key the data does not keep unique can make Except remove too much, never return a row no set admitted.
- A type with no primary key (a keyless entity type, or a query EF Core does not map to T) is combined with SQL
`UNION` / `INTERSECT` / `EXCEPT`, which compare whole rows:
- identical rows collapse into one;
- every mapped column must be comparable, even when unselected: not PostgreSQL `json`, SQL Server `xml` or spatial
types;
- the provider must support the operators the request uses (MySQL has `INTERSECT` and `EXCEPT` from 8.0.31).
- `Orders` apply before the projection, as for `Filter`, so an order field need not be selected. Order is the
database's: text sorts by collation and NULLs fall where the provider puts them.
- Under `ApplyPolicy`, `Caps.MaxConditionSets` (default 10) bounds how many sets one statement carries (section 12).
- Validation: duplicate set Sort → `ListOfConditionsSetsMustHasUniqueSortValue`; a set after the first with a
null `Intersection` → `ConditionsSetOfIndex[1-N]MustHasIntersection`; the first set's `Intersection` is ignored;
a set with a null `ConditionGroup` → `ArgumentNullException`. Every clause is validated before the database is
queried.
- Empty or null `ConditionSets` runs `ToListAsync(new Filter { Selects, Orders, Page })`: there is nothing to combine.
- No synchronous version, no `getQueryString`; needs an EF Core async provider. An overload takes a
`CancellationToken` (3.2.0) and passes it to the count and the read. Only `Except` on a type with a
primary key needs the provider to translate a correlated `EXISTS`; `Union` and `Intersect` there are plain `OR` and
`AND`.
### In memory: the IEnumerable<T> overloads
- `ToList(Filter)`, `ToListDynamic(Filter)` and `ToList(Summary)` on `IEnumerable<T>` call `AsQueryable()` and
run the same pipeline with LINQ to Objects. For any other method call `list.AsQueryable()` yourself; the async
methods do not work on such a source.
- Differences from EF Core: text operators follow .NET string semantics; a null reference navigation in a path
throws `NullReferenceException`; typed `Select` through a reference navigation throws (EF.Property);
ordering by a reference navigation throws; `getQueryString` returns the "does not support generation of query
strings" text.
---
## 7. Grouping and aggregation — Group<T>, Summary<T>, ToList(Summary)
```
Summary<T> validate -> Where(ConditionGroup) -> GroupBy + aggregates -> Having -> order -> Skip/Take -> IQueryable
ToList, ToListAsync validate -> Where -> GroupBy + aggregates -> Having -> COUNT(groups) -> order -> Skip/Take -> SummaryResult
Group<T> validate GroupBy -> GroupBy + aggregates -> IQueryable
```
```
GroupBy.Fields emitted row properties
["IsActive"] GroupBy("IsActive").Select("new (Key as IsActive, …)") IsActive
["CreatedAt.Year"] GroupBy("CreatedAt.Year").Select("new (Key as CreatedAtYear, …)") CreatedAtYear
["IsActive", "Category.Name"] GroupBy("new (IsActive, Category.Name)")
.Select("new (Key.IsActive as IsActive, Key.Name as CategoryName, …)")
IsActive, CategoryName
```
- Each row has one property per group field, named by the normalized path with the dots removed, holding the
key's own type (an enum key stays an enum), then one property per `AggregateBy.Alias`, in list order. Rows are
`DynamicClass` objects; `SummaryResult.Data` is `List<dynamic>`.
- A null group key is its own group (`{ "categoryName": null, … }`).
- `ToListAsync(Summary)` counts and reads through EF Core's `CountAsync` and `ToListAsync` (3.2.0; it used to count
synchronously). On a provider that is not EF Core's, rows in memory among them, it counts and reads synchronously.
```
Aggregator Emitted per group Field accepted Result type
Count Count() optional; validated if given, unused int
CountDistinct Select(f).Distinct().Count() any simple type int
Sumation Sum(f) numeric as Sum(f): int -> int, decimal -> decimal
Average Average(f) numeric as Average(f): int -> double, decimal -> decimal
Minimum Min(f) simple, not bool / bool? the field's type
Maximum Max(f) simple, not bool / bool? the field's type
FirstOrDefault Select(f).OrderBy($).FirstOrDefault() any simple type the field's type: the SMALLEST value
LastOrDefault Select(f).OrderByDescending($).FirstOrDefault() any simple type the field's type: the LARGEST value
```
- **numeric** = byte, sbyte, short, ushort, int, uint, long, ulong, float, double, decimal and their nullable forms.
- **simple** = any primitive, string, decimal, DateTime, DateOnly, TimeOnly, DateTimeOffset, TimeSpan, Guid, enum,
and their nullable forms.
- **Alias** must be an identifier: a letter (any script) or `_`, then letters, digits or `_`. Valid: `Total_Sales`,
`Total2`, `المجموع`. Invalid (`AggregationMustHasValidAlias`): `Total Sales`, `Total-Sales`, `Total.Sales`,
`1Total`, `""`. The Alias is checked before the Field, so an entry without an Alias always fails on the Alias.
- **Group and aggregate fields** must end on a simple type. A navigation, or a collection of entities, throws
`GroupByFieldCannotBeComplexType` / `AggregationFieldMustBeSimpleType` (the element type is what is checked, so the
`…CannotBeCollectionType` strings fire only for a collection of collections). A path through a collection to a
scalar, or a collection of simple values, passes validation but gets no Any() / Select, so do not group or
aggregate across a collection.
- **Key names clash silently.** Inside a multi-field key, members are named by their last segment: two group fields
ending in the same segment (`"Name"`, `"Category.Name"`) pass validation and throw
`InvalidOperationException("Sequence contains more than one matching element")` at run time. An Alias equal to a
dot-stripped group field (`CategoryName` beside `Category.Name`) is not refused either — the alias check compares
against the dotted path — and one of the two columns silently disappears from the rows.
- **Having.** `Summary.Having` is a `ConditionGroup` over the grouped rows. Each condition's Field must be an
Alias (case-insensitive), else `HavingField[<field>]MustExistInAggregateByAliases`; a group field is not allowed.
Value count, value format, Sort uniqueness, the null guard and the DataType/Operator table work as in a where.
With no `AggregateBy`, every Having condition fails.
- **Summary.Orders.** Each Field must be a group field — dotted (`Category.Name`) or with the dots removed
(`CategoryName`) — or an Alias, case-insensitive; else
`SummaryOrderField[<field>]MustExistInGroupByFieldsOrAggregateByAliases`. Emitted as `<field without dots> asc|desc`,
sorted by Sort, with no collection rewriting and no duplicate-Sort check.
- Summary validation order: `GroupBy` null (`ArgumentNullException`, parameter `GroupBy`) → GroupBy and AggregateBy
rules → Orders → Page → Having → ConditionGroup → Having DataType/Operator pairs.
- Aggregation runs in SQL on stored values. Under `ApplyPolicy`, transformed fields need `AllowAggregate`, every
guarded summary is subject to the group floor (section 19), and `Caps.MaxAggregates` (default 50, 3.1.0) bounds how
many `AggregateBy` entries one summary sends (section 12).
---
## 8. Validation and core error strings
A broken rule throws `LogicException` (`DynamicWhere.ex.Exceptions`). The error string is `Message`; there is no
separate code property. Two constructors: `LogicException(string message)` and, since 3.1.0,
`LogicException(string message, string? subject)`, whose `Subject` carries what the refusal is about — a type
name, a field — so the message stays one of the fixed strings a caller matches on. Under `ApplyPolicy` a field is
named as the caller wrote it, so a `[DwAlias]` name is never replaced by the member behind it. `PolicyException`
derives from it, so catch `PolicyException` first.
### Every core error string
```
String Raised when
ConditionMustHasValidFieldName a Condition / OrderBy / GroupBy / AggregateBy / Having / Summary.Orders
field is null or blank, or a path does not resolve on T. Under
ApplyPolicy in the Strict tier, outside dry run, an unresolved
path is a PolicyException instead (3.1.0, section 17)
FieldPath[<path>]StartsWithReservedName a Condition / OrderBy / GroupBy / AggregateBy / Selects path, or a
[DwAlias] target, whose first segment is one of the parser's own
words — new, iif, np, isnull, is, as, cast, true, false, null,
whatever the letter case. Subject = that segment, trimmed. Not a
member of ErrorCode: the string is built where the path is
validated. 3.1.0, section 5
ConditionWithOperator[<Operator>]MustHasOnlyOneValue an operator other than those below has 0 or 2+ values
ConditionWithOperator[Between-NotBetween]MustHasOnlyTwoValues Between / NotBetween without exactly 2 values
ConditionWithOperator[In-IIn-NotIn-INotIn]MustHasOneOrMoreValues an In-family operator with no values
ConditionWithOperator[IsNull-IsNotNull]MustHasNoValues IsNull / IsNotNull with any value
InvalidFormat a Guid / Number / Boolean / Date / DateTime value fails its TryParse
AnyListOfConditionsMustHasUniqueSortValue two Conditions of one group (or Having group) share Sort
AnyListOfSubConditionsGroupsMustHasUniqueSortValue two SubConditionGroups of one group share Sort
ListOfConditionsSetsMustHasUniqueSortValue two ConditionSets share Sort
ConditionsSetOfIndex[1-N]MustHasIntersection a ConditionSet after the first (by Sort) has Intersection null
OrderField[<field>]CannotEndOnCollectionOfComplexElements an order path ends on a collection of entities
PageNumberMustBeGreaterThanZero PageNumber <= 0
PageSizeMustBeGreaterThanZero PageSize <= 0
MustHasFields Select / SelectDynamic / Selects given an empty list
GroupByMustHasAtLeastOneField GroupBy.Fields empty
GroupByFieldsMustBeUnique a group field repeated (case-insensitive)
GroupByFieldCannotBeComplexType a group field ends on a navigation or entity collection
GroupByFieldCannotBeCollectionType a group field ends on a collection of collections
AggregationMustHasValidAlias Alias is not an identifier
AggregationFieldMustBeSimpleType an aggregate field ends on a navigation or entity collection
AggregationFieldCannotBeCollectionType an aggregate field ends on a collection of collections
Aggregator[<Aggregator>]IsNotSupportedForFieldType[<TypeName>] Sumation / Average on a non-numeric type, Minimum / Maximum on bool;
<TypeName> is Type.Name, so a bool? member shows Nullable`1
AggregationAliasesMustBeUnique an Alias repeated (case-insensitive)
AggregationAlias[<alias>]CannotBeUsedInGroupByFields an Alias equals a group field path (case-insensitive)
SummaryOrderField[<field>]MustExistInGroupByFieldsOrAggregateByAliases a Summary order field is neither a group field nor an Alias
HavingField[<field>]MustExistInAggregateByAliases a Having field is not an Alias
ConditionValuesAreNullOrWhiteSpace defined but never thrown
AmbiguousDateFormat a Date / DateTime value that leads with a day or a month
("01/09/2026") and matches no declared format, or one two
accepted formats read differently (section 4, Date formats).
Subject = the field. 3.1.0
SelectTypeMustHaveParameterlessConstructor Select<T> / Filter.Selects on a T with no parameterless
constructor — a positional record, most often. Also a
guarded query where a member carries [DwNoSelect],
because deny-select projects. Subject = T.Name.
3.1.0; before it, an English sentence
Unsupported combination of DataType '<DataType>' and Operator '<Operator>'. a pair outside the section 4 table
```
The last one is a literal message; every other string above is a fixed code. Examples: `ConditionWithOperator[Equal]MustHasOnlyOneValue`,
`Aggregator[Sumation]IsNotSupportedForFieldType[String]`, `HavingField[UnitPrice]MustExistInAggregateByAliases`.
### Other exceptions a caller can see
```
ArgumentNullException a null query or shape argument; a null or blank Selects entry (parameter "name");
Summary.GroupBy null (parameter "GroupBy"); a ConditionSet whose ConditionGroup is null
NullReferenceException a null element inside Conditions, SubConditionGroups, ConditionSets or Orders;
in memory, a null reference navigation inside a path
PolicyException PolicyRequired from the [DwEntity(RequirePolicy = true)] guard; every refusal under ApplyPolicy
ParseException System.Linq.Dynamic.Core.Exceptions.ParseException for input that passes validation but not
the parser: "1,000", "NaN", an enum name that is not a member, Contains on an enum-typed
member, a Where on a collection of simple values
InvalidOperationException async terminal on a non-EF source; typed Select through a reference navigation in memory;
two group fields ending in the same segment; ordering by a non-comparable type in memory
EF Core / provider exceptions unwrapped (for example SQLite NotSupportedException for ORDER BY on decimal)
```
- The engine catches nothing; everything reaches the caller unwrapped.
- Timing: composable methods throw at the call; async terminals at `await`; the dynamic terminals run the COUNT
query before validating Orders, Page and Selects. A Segment validates every set and clause before its first query
(3.1.0).
### Checks by shape, in order
```
Condition Field null/blank → Field resolves → value count → value format → DataType/Operator pair
ConditionGroup Sort unique among Conditions → Sort unique among SubConditionGroups → each Condition in Sort order →
each sub-group in Sort order (recursively)
OrderBy Field null/blank → Field resolves → not ending on a collection of entities
PageBy PageNumber >= 1 → PageSize >= 1
Selects list not null (ArgumentNullException) → not empty → no blank entry → each resolves → constructor (Select<T>)
GroupBy Fields not empty → each field: null/blank, resolves, unique, not complex → each AggregateBy:
Alias identifier → Field (unless Count) → simple type → aggregator valid for type → Alias not a group
field → Alias unique
Summary GroupBy not null → GroupBy rules → Orders → Page → Having (count, format) → ConditionGroup → Having pairs
Segment set Sort unique → Intersection on later sets → each set's ConditionGroup → Orders → Page → Selects
Filter ConditionGroup → Orders → Page → Selects, each only when not null
```
---
## 9. JSON wire format
The shapes and results carry no JSON attributes or converters, and the query path serializes nothing: the host's
serializer binds the request and writes the result.
```
JSON body Bind to Pass to
["Id", "Category.Name"] List<string> Select, SelectDynamic
{ sort, field, dataType, operator, values } Condition Where
{ sort, connector, conditions, subConditionGroups } ConditionGroup Where
{ sort, field, direction } / [ … ] OrderBy / List<OrderBy> Order
{ pageNumber, pageSize } PageBy Page
{ fields, aggregateBy: [ { field, alias, aggregator } ] } GroupBy Group
{ conditionGroup, selects, orders, page } Filter Filter, FilterDynamic, ToList, ToListAsync,
ToListDynamic, ToListAsyncDynamic
{ conditionGroup, groupBy, having, orders, page } Summary Summary, ToList, ToListAsync
{ conditionSets: [ { sort, intersection, conditionGroup } ],
selects, orders, page } Segment ToListAsync
```
### Property names
- Keys are the C# property names. ASP.NET Core's defaults (`JsonSerializerDefaults.Web`) bind them
case-insensitively, so `conditionGroup` and `ConditionGroup` both work, and write camelCase.
- Path strings (`field`, `fields`, `selects`) are case-insensitive per segment and trimmed; they are rewritten to
the exact CLR names, and results use the rewritten names.
- `alias` becomes an output member name exactly as sent.
### Enums need a converter for names
The library ships no enum converter. Without one, `System.Text.Json` refuses enum names with `JsonException`
("The JSON value could not be converted to DynamicWhere.ex.Enums.Connector") — a 400 in ASP.NET Core. Either send
the numbers of section 3, or register `JsonStringEnumConverter`, which then accepts names in any case
(`"IContains"`, `"icontains"`) as well as numbers:
```csharp
// MVC controllers
builder.Services.AddControllers().AddJsonOptions(o =>
o.JsonSerializerOptions.Converters.Add(new JsonStringEnumConverter()));
// minimal APIs
builder.Services.Configure<Microsoft.AspNetCore.Http.Json.JsonOptions>(o =>
o.SerializerOptions.Converters.Add(new JsonStringEnumConverter()));
```
The same setting decides how results write enums: `policy.tier`, `decisions[].feature`, `decisions[].action`,
and enum-typed members and group keys in `data`.
### Omitted and empty values
- An omitted enum property silently takes member 0: `dataType` Text, `operator` Equal, `connector` And,
`direction` Ascending, `aggregator` Count. An omitted `sort` is 0.
- Omitted or null `values` means `[]`. An `aggregateBy` entry may omit `field` only for Count.
- In Filter, Summary and Segment an omitted or null `conditionGroup`, `selects`, `orders`, `page` or `having` is
skipped; an omitted `intersection` is null.
- `"selects": []` throws `MustHasFields` — omit the key instead. `"fields": []` throws `GroupByMustHasAtLeastOneField`.
- `"orders": []` and a group with no conditions change nothing. `"conditionSets": []` runs the segment as a plain filter.
Under `ApplyPolicy` a Filter or Segment with no orders takes the type's declared `DefaultOrder`, if any
(3.1.0, section 13); unguarded, no orders means no ordering.
- Inside a Segment, a set whose group has no conditions stands for every row: a `Union` with it returns every row, an
`Intersect` with it changes nothing, and an `Except` of it returns nothing.
- A Summary with no `groupBy` throws `ArgumentNullException`, not `LogicException`.
- A `null` inside `values` is `""`, not SQL NULL (section 4).
### Results as JSON
```jsonc
{
"pageNumber": 1, // 0 when the request had no page, unless DwCaps.DefaultPageSize gave a guarded query one
"pageSize": 10, // 0 when the request had no page, with the same exception
"pageCount": 5, // no page: 1, or 0 with no rows
"totalCount": 42, // before paging
"data": [ ],
"queryString": null, // SQL only with getQueryString: true
"policy": null // ApplyPolicy terminals only; under the Strict tier only with IncludeTraceInResult (3.1.0):
// { "tier": "Convenience", "dryRun": false,
// "decisions": [ { "fieldPath": "Email", "feature": "Select", "action": "Masked", "reason": "Mask" } ] }
}
```
```
data rows by method
ToList, ToListAsync (Filter) T with every member. With selects, unselected members hold defaults (0, "", null,
ToListAsync (Segment) "0001-01-01T00:00:00"); a selected reference navigation also carries its Id and is an
empty object, not null, when missing
ToListDynamic, ToListAsyncDynamic no selects: the entity. With selects: only what was asked, nested by path —
"Category.Name" -> { "category": { "name": … } }, "OrderItems.Quantity" ->
{ "orderItems": [ { "quantity": … } ] }; no Id added
ToList, ToListAsync (Summary) one flat row per group: each group field with dots removed ("Category.Name" ->
categoryName), then each alias
```
- Typed rows and `DynamicClass` rows (dynamic filters, unguarded summaries) are objects with properties, so the
host naming policy applies: camelCase under ASP.NET Core defaults (`CategoryName` → `categoryName`).
- Under `ApplyPolicy`, a dynamic or summary row is rebuilt as an `ExpandoObject` when it carries a `[DwAlias]`
column or the group floor applied (every guarded `ToList(Summary)` with the default floor). System.Text.Json writes
`ExpandoObject` keys as they are, so those rows keep PascalCase and alias spelling (`{ "Name": "Ann", "dept": "Eng" }`)
while the envelope is camelCase.
---
## 10. JSON recipes
Field names follow this model:
```
Product Id:Guid Name:string Price:decimal Rating:double StockQuantity:int IsActive:bool
CreatedAt:DateTime UpdatedAt:DateTime? Tags:List<string> CategoryId:Guid?
Category:Category? OrderItems:ICollection<OrderItem> Reviews:ICollection<Review>
Category Id:Guid Name:string ParentCategory:Category?
OrderItem Id:Guid Quantity:int ProductId:Guid Product:Product
Review Id:Guid Rating:int
Order Id:Guid Status:OrderStatus TotalAmount:decimal CustomerId:Guid OrderItems:ICollection<OrderItem>
Customer Id:Guid Orders:ICollection<Order>
OrderStatus Pending Confirmed Processing Shipped Delivered Cancelled Refunded
```
### Projection — Select, SelectDynamic
```json
["Id", "Name", "Category.Name", "OrderItems.Quantity"]
```
```
Select<Product> Product { Id, Name, Category { Id, Name }, OrderItems [ { Id, Quantity } ] }, every other member default
SelectDynamic<Product> { Id, Name, Category: { Name }, OrderItems: [ { Quantity } ] }
```
A bare path list is the body of `Select` / `SelectDynamic`; inside a Filter or Segment the same list goes in `selects`.
### Where — one Condition per DataType
```jsonc
{ "sort": 1, "field": "Name", "dataType": "Text", "operator": "IContains", "values": ["pro"] }
{ "sort": 1, "field": "Price", "dataType": "Number", "operator": "Between", "values": [10, 500.5] }
{ "sort": 1, "field": "IsActive", "dataType": "Boolean", "operator": "Equal", "values": [true] }
{ "sort": 1, "field": "CreatedAt", "dataType": "Date", "operator": "GreaterThanOrEqual", "values": ["2024-01-01"] }
{ "sort": 1, "field": "CreatedAt", "dataType": "DateTime", "operator": "LessThan", "values": ["2024-06-15T14:30:00"] }
{ "sort": 1, "field": "UpdatedAt", "dataType": "DateTime", "operator": "IsNull", "values": [] }
{ "sort": 1, "field": "CategoryId", "dataType": "Guid", "operator": "In", "values": ["a1b2c3d4-e5f6-7890-abcd-ef1234567890"] }
{ "sort": 1, "field": "Status", "dataType": "Enum", "operator": "In", "values": ["Pending", "Shipped"] } // on Order
```
Each line is a separate `Condition` body. The Between line becomes `(Price != null && Price >= 10 && Price <= 500.5)`.
### Where — nested AND / OR
```json
{
"connector": "And",
"conditions": [
{ "sort": 1, "field": "IsActive", "dataType": "Boolean", "operator": "Equal", "values": [true] }
],
"subConditionGroups": [
{
"sort": 1,
"connector": "Or",
"conditions": [
{ "sort": 1, "field": "Price", "dataType": "Number", "operator": "LessThan", "values": [20] },
{ "sort": 2, "field": "Rating", "dataType": "Number", "operator": "GreaterThanOrEqual", "values": [4.5] }
]
}
]
}
```
`IsActive AND (Price < 20 OR Rating >= 4.5)`.
### Where — a path through collections (on Customer)
```json
{ "sort": 1, "field": "Orders.OrderItems.Product.Name", "dataType": "Text", "operator": "IContains", "values": ["laptop"] }
```
`(Orders.Any(i1 => i1.OrderItems.Any(i2 => i2.Product.Name != null && i2.Product.Name.ToLower().Contains("laptop"))))`
— a customer matches when some order has some item whose product name contains "laptop".
### Order — several keys, across collections
```json
[
{ "sort": 1, "field": "Category.Name", "direction": "Ascending" },
{ "sort": 2, "field": "Reviews.Rating", "direction": "Descending" },
{ "sort": 3, "field": "OrderItems.Product.Name" }
]
```
`Category.Name asc, Reviews.Select(Rating).DefaultIfEmpty().Max() desc, OrderItems.Min(Product.Name) asc`.
### Page
```json
{ "pageNumber": 3, "pageSize": 25 }
```
Skip 50, take 25.
### Filter — typed and dynamic
```json
{
"conditionGroup": {
"connector": "And",
"conditions": [
{ "sort": 1, "field": "Price", "dataType": "Number", "operator": "GreaterThan", "values": [50] },
{ "sort": 2, "field": "Category.Name", "dataType": "Text", "operator": "IEqual", "values": ["smartphones"] }
]
},
"selects": ["Id", "Name", "Price", "Category.Name"],
"orders": [{ "sort": 1, "field": "Price", "direction": "Descending" }],
"page": { "pageNumber": 1, "pageSize": 10 }
}
```
```
ToListAsync FilterResult<Product>: data[i] = Product { Id, Name, Price, Category { Id, Name } }, everything else default
ToListAsyncDynamic FilterResult<dynamic>: data[i] = { Id, Name, Price, Category: { Name } }
```
### Summary — group, aggregate, having, order by alias
```json
{
"conditionGroup": {
"connector": "And",
"conditions": [{ "sort": 1, "field": "IsActive", "dataType": "Boolean", "operator": "Equal", "values": [true] }]
},
"groupBy": {
"fields": ["Category.Name"],
"aggregateBy": [
{ "alias": "ProductCount", "aggregator": "Count" },
{ "field": "Price", "alias": "AvgPrice", "aggregator": "Average" },
{ "field": "Price", "alias": "Revenue", "aggregator": "Sumation" }
]
},
"having": {
"connector": "And",
"conditions": [{ "sort": 1, "field": "ProductCount", "dataType": "Number", "operator": "GreaterThan", "values": [5] }]
},
"orders": [{ "sort": 1, "field": "Revenue", "direction": "Descending" }],
"page": { "pageNumber": 1, "pageSize": 10 }
}
```
`SummaryResult`: `data[i] = { CategoryName, ProductCount, AvgPrice, Revenue }`; `totalCount` = groups left after
having. `Group<T>` takes the same `groupBy` object and returns the rows without having, order or page.
### Segment — Union, Intersect, Except
```json
{
"conditionSets": [
{ "sort": 1, "conditionGroup": { "conditions": [
{ "sort": 1, "field": "IsActive", "dataType": "Boolean", "operator": "Equal", "values": [true] }] } },
{ "sort": 2, "intersection": "Union", "conditionGroup": { "conditions": [
{ "sort": 1, "field": "Price", "dataType": "Number", "operator": "GreaterThan", "values": [100] }] } },
{ "sort": 3, "intersection": "Except", "conditionGroup": { "conditions": [
{ "sort": 1, "field": "StockQuantity", "dataType": "Number", "operator": "Equal", "values": [0] }] } }
],
"orders": [{ "sort": 1, "field": "Price", "direction": "Descending" }],
"page": { "pageNumber": 1, "pageSize": 20 }
}
```
`SegmentResult<Product>`: (active UNION price > 100) EXCEPT out-of-stock, combined into one query and ordered and
paged in the database. `selects` can be added; it projects the page, as for a filter (section 6).
---
## 11. Traps — query engine
Each of these compiles, passes validation, and returns something other than what was meant.
1. **A `Segment` over a type with no primary key compares whole rows.** A keyed entity combines by row. A keyless
entity type, or a query EF Core does not map to T, uses SQL `UNION` / `INTERSECT` / `EXCEPT`: identical rows
collapse into one, and a column the database cannot compare (PostgreSQL `json`, SQL Server `xml`) fails the
query even when it is not selected. Give the type a key, or map it without such columns.
2. **Typed `Select` still returns whole T objects.** Unselected members hold defaults and a serializer writes all of
them (`"price": 0`, `"createdAt": "0001-01-01T00:00:00"`, `"category": {}`). Use `ToListDynamic` for a payload
holding only the selected members.
3. **Negated operators never match null.** `NotEqual`, `NotIn`, `NotContains`, `NotStartsWith`, `NotEndsWith` and
`NotBetween` all exclude rows whose member is null. Add an `IsNull` condition in an `Or` group to keep them.
4. **Validation rewrites the request objects you pass.** Paths are re-cased, null lists become empty, the first
set's `Intersection` is cleared. Clone a shape before reusing it. (Guarded calls work on a clone.)
5. **Enum names in JSON need `JsonStringEnumConverter`.** Without it the body fails to bind. An omitted enum
property silently becomes member 0: `Text`, `Equal`, `And`, `Ascending`, `Count`.
6. **A `DateTime` member reads a zoned value as server local time.** A value carrying `Z` or an offset is converted
to the host's local time before comparing against a `DateTime` column; a `DateTimeOffset` column normalises to
UTC instead. Send ISO 8601 in the convention the column stores. (Before 3.1.0 this trap was worse: dates were
parsed in the server's culture, so `"01/02/2024"` meant different days on different servers. A day/month-first
date is now refused with `AmbiguousDateFormat` unless the deployment declares its order.)
7. **`FirstOrDefault` and `LastOrDefault` return the minimum and maximum value**, not the first and last row.
8. **Each `Order(...)` call replaces the previous ordering.** Put every key in one `List<OrderBy>`.
9. **Some requests pass validation and then fail in the parser** with `System.Linq.Dynamic.Core.Exceptions.ParseException`:
`Contains` / `StartsWith` / `EndsWith` on an enum-typed member, a `Where`
directly on a `List<string>`, a Number value with a thousands separator or `NaN`, an enum name that is not a member.
Treat `ParseException` as a 400 too.
10. **Summary column names collide silently.** Two group fields ending in the same segment (`Name`, `Category.Name`)
throw `InvalidOperationException` at run time, and an alias equal to a dot-stripped group field (`CategoryName`
beside `Category.Name`) silently drops a column.
11. **In-memory sources behave differently from EF Core.** The async Filter and Segment terminals throw; typed `Select` through a
reference navigation throws; a null navigation inside a path throws `NullReferenceException`; ordering by a
navigation throws; `getQueryString` returns a placeholder sentence instead of SQL.
12. **`ToListAsync(filter, default)` is ambiguous (3.2.0).** `default` fits both `getQueryString` and the new
`CancellationToken` overload, and the call does not compile. Write `false`, a token, or a named argument.
13. **`Selects: []` throws `MustHasFields`.** Omit the property (null) to return whole entities.
14. **A null inside `Values` is the empty string, not NULL.** Test for NULL with `IsNull` / `IsNotNull` and no values.
15. **`Page` does not order.** Paging without `Orders` returns whatever order the database chooses; always send an
order with a page. `[DwEntity(DefaultOrder = ...)]` changes that only for a guarded query (3.1.0, section 13).
16. **Validation is not one pass before the query.** Each clause is checked as it is composed, so some invalid input is
refused only after the database has been hit: `ToListDynamic` / `ToListAsyncDynamic` run the `COUNT` query before
`Orders`, `Page` and `Selects` are validated. The `LogicException` is the same one either way; the round trip is
not undone.
---
## 12. Policy lifecycle and configuration
Sections 12–26 are all in the core package `DynamicWhere.ex`; sections 27–29 are the companion packages.
A project with no policy attributes and no `ApplyPolicy` call behaves exactly as 2.x.
### Lifecycle
```csharp
using DynamicWhere.ex.Policies.Config; // DwPolicy, DwPolicyOptions, AddDwPolicies
using DynamicWhere.ex.Policies.Context; // DwPolicyContext
using DynamicWhere.ex.Policies.Enums; // DwTier, DwSubjectKind
using DynamicWhere.ex.Policies.Source; // ApplyPolicy
using DynamicWhere.ex.Policies.Tokens; // InMemoryTokenVault
// 1. Startup, exactly once. Both forms end in DwPolicy.Configure, which freezes the options.
builder.Services.AddDwPolicies(
builder.Configuration.GetSection("DynamicWhere:Policies"),
options =>
{
options.Entities.Expose<Employee>("Employee");
options.TokenVault = new InMemoryTokenVault();
});
// or, without configuration:
DwPolicy.Configure(new DwPolicyOptions { Tier = DwTier.Strict, HashSalt = secret }, providers);
// 2. Once per request: build the whole caller, prepare, then query.
DwPolicyContext caller = await DwPolicy.PrepareAsync(
new DwPolicyContext { Purpose = "support" }
.WithSubject(DwSubjectKind.User, userId)
.WithSubject(DwSubjectKind.Role, "Support")
.WithSubject(DwSubjectKind.Tenant, tenantId)
.WithValue("TenantId", tenantId));
// 3. Per query.
FilterResult<Employee> result = await db.Employees.ApplyPolicy(caller).ToListAsync(filter);
// 4. Once per request, after the queries, when any field carries [DwAudit] or AuditRefusals is on.
await DwPolicy.DrainAuditAsync(caller, sink);
```
What one guarded call does, in order:
```
Strict getQueryString check → clone the request → count caps → resolve names and aliases → MaxNavigationDepth
→ cost budget (Convenience, and any dry run) → gate each field → default order when no orders were sent
(Filter, Segment) → cost budget (Strict outside dry run, 3.1.0) → inject forced predicates
→ check required filters → group floor (Summary) → unchanged core method over AsNoTracking()
→ transform materialized values → rename aliased columns (dynamic and Summary rows)
→ result.Policy = trace when IncludeTraceInResult allows it (by default: Convenience yes, Strict no)
a refusal at any step: PolicyException, also written to the audit buffer when AuditRefusals is on
```
- Nothing is mandatory.
- Before `Configure`, `ApplyPolicy` still enforces attributes. It uses a frozen default `DwPolicyOptions`: `Convenience` tier, default caps, `MinGroupSize` 5, no store.
- Unguarded calls on `DynamicWhere.ex.Source.Extension` are never sanitized, capped, transformed, traced or given a `DefaultOrder`.
- The only policy check they run is `[DwEntity(RequirePolicy = true)]`.
- The first failing check throws.
- The count caps (`CapExceeded`) run before any name is resolved (3.1.0), so they win over everything after them, a name that matches nothing or is ambiguous included. `MaxNavigationDepth` runs once names are resolved.
- In the Convenience tier and in dry run the cost budget (`QueryCostExceeded`) wins over field denials. Under `Strict`, outside dry run, the budget is checked after every field gate (3.1.0), so a field denial wins over it.
- A field path that names nothing on `T` throws `LogicException` from validation, before any policy decision: unguarded, in the Convenience tier, and in dry run. Under `ApplyPolicy` the count caps run first.
- In the Strict tier (3.1.0), outside dry run, it is kept and gated as a field denied for every feature, after the caps, so it is refused exactly as a denied field is (section 17).
- The `Filter`, `Summary` or `Segment` passed in is never modified. Each guarded call works on a clone.
- `PrepareAsync` reads nothing unless a `StorePolicyProvider` is configured, but it always records that it ran:
`DwPolicyContext.IsPrepared` (3.1.0).
- Since 3.1.0 `ApplyPolicy(ctx)` — the overloads that read `DwPolicy` — refuse a context that never went through
`PrepareAsync` with `PolicyContextNotPrepared` (18), store or no store. Before 3.1.0 only a store provider did,
so an attributes-only deployment accepted an unprepared context and would start refusing the day it gained one.
- The overload taking explicit `DwPolicyOptions` and `PolicyResolver` does not check: that host owns preparation,
and a store it hands in still refuses an unprepared context itself.
- A store provider's own two refusals sit behind that check: no attachment, and a `User` subject added after
preparation.
- The simulator's copy of a prepared context is prepared too.
### DwPolicy
```
DwPolicy static class
Options : DwPolicyOptions frozen; a frozen default instance until Configure
IsConfigured : bool
Resolver : PolicyResolver AttributePolicyProvider + configured providers
StoreProviders : IReadOnlyList<StorePolicyProvider> configured store providers, in the order supplied
Configure(DwPolicyOptions options, params IDwPolicyProvider[] providers) -> void
PrepareAsync(DwPolicyContext context, CancellationToken ct = default) -> ValueTask<DwPolicyContext>
DrainAuditAsync(DwPolicyContext context, IDwAuditSink sink, CancellationToken ct = default) -> ValueTask<int>
ValidateModel(params Type[] types) -> PolicyModelReport
ValidateModel(DwPolicyOptions? options, params Type[] types) -> PolicyModelReport
```
`DwPolicy` has no `Reset`, `Explain` or `IsPrepared`. For isolation (tests, custom hosts), use the four-argument `ApplyPolicy`.
- `Configure` refuses only two things:
- null `options` → `ArgumentNullException`;
- a second call, including one made by `AddDwPolicies` → `InvalidOperationException`.
- `Configure` freezes `options`, `options.Caps` and `options.Entities`. Any setter or `Expose` after that → `InvalidOperationException`.
- `Configure` does not validate the model.
- Invalid values were already refused by the setters.
- A missing `HashSalt` or `TokenVault` only surfaces at query time (`MissingHashSalt` 21 / `MissingTokenVault` 22), unless `ValidateModel(options, types)` ran first.
- `AttributePolicyProvider` is always added first. Passing one yourself is ignored, null elements are skipped, and a null array means none.
- `DwPolicy.Options` is already frozen before `Configure`. Build a new `DwPolicyOptions`; never mutate `DwPolicy.Options`.
- `PrepareAsync`:
- null → `ArgumentNullException`;
- for each store provider, in order, it pins that provider's current snapshot to the context and loads the rules for every `User` subject;
- a load failure propagates;
- it returns the same instance.
- `DrainAuditAsync`:
- null context or sink → `ArgumentNullException`;
- it removes every buffered event, writes them in order, and returns how many were written;
- if the sink throws, the failing event and everything after it go back to the front of the buffer and the exception is rethrown;
- there is no retry.
- `ValidateModel` runs `PolicyModelValidator.Inspect`.
- Any error → `InvalidOperationException` whose message lists every error.
- Otherwise it returns the report, warnings included.
- Pass the options you will `Configure` with, or the salt and vault checks are skipped.
### AddDwPolicies and Bind
```
DwPolicyConfiguration static class
AddDwPolicies(this IServiceCollection services, IConfiguration section,
Action<DwPolicyOptions>? configure = null,
params IDwPolicyProvider[] providers) -> IServiceCollection
Bind(this DwPolicyOptions options, IConfiguration section) -> DwPolicyOptions same instance
```
`AddDwPolicies` does exactly this: `new DwPolicyOptions().Bind(section)` → `configure?.Invoke(options)` → `DwPolicy.Configure(options, providers)` → `services.AddSingleton(options)`.
- There is one overload, and `section` is required (null → `ArgumentNullException`).
- `"DynamicWhere:Policies"` is a convention; any section works.
- For configuration from code only, call `DwPolicy.Configure`.
- Configuration binds first and `configure` runs second, so code wins over the file.
- It configures the static `DwPolicy` immediately, during service registration. Anything set in `configure` (`Services` included) must already exist at that point.
- It registers only the frozen `DwPolicyOptions` singleton. No resolver, sink, vault, catalogue or provider is registered.
- A `StorePolicyProvider` reads `StoreFailure`, `MaxSnapshotAge` and `RefreshInterval` from the options instance passed to `StorePolicyProvider.CreateAsync`.
- `AddDwPolicies` creates its options internally, so a provider cannot share them. With a store, bind by hand:
```csharp
DwPolicyOptions options = new DwPolicyOptions().Bind(builder.Configuration.GetSection("DynamicWhere:Policies"));
options.Entities.Expose<Employee>("Employee");
StorePolicyProvider store = await StorePolicyProvider.CreateAsync(policyStore, options);
DwPolicy.Configure(options, store);
builder.Services.AddSingleton(options);
```
- `options.Bind(section)` is the DynamicWhere extension and binds with `ErrorOnUnknownConfiguration = true`.
- `section.Bind(options)` is Microsoft's binder and silently ignores unknown keys. Do not use it.
- A key that no property answers to (`Caps:MinGropSize`, `Teir`) → `InvalidOperationException` at bind time.
- A value its setter refuses also fails the bind:
- a cap below its minimum;
- a `HashSalt` of 1–15 characters;
- a non-positive `MaxSnapshotAge` or `RefreshInterval`.
- Binding onto frozen options fails.
- An empty or missing section leaves every default in place, and `Caps.IsMinGroupSizeSet` stays false.
- `TokenVault`, `Services` and `Entities` are objects, not values. They cannot come from configuration; set them in `configure`.
### Configuration keys
```
Tier "Convenience" | "Strict"
DryRun true | false
IncludeTraceInResult true | false; leave it out to follow the tier (3.1.0)
AuditRefusals true | false (3.1.0)
HashSalt string; supply via user secrets, an environment variable or a vault, never a committed file
StoreFailure "LastKnownGood" | "FailClosed" | "StaticOnly"
MaxSnapshotAge TimeSpan "hh:mm:ss", e.g. "00:15:00"
RefreshInterval TimeSpan "hh:mm:ss", e.g. "00:00:30"
Caps:MaxPageSize Caps:DefaultPageSize Caps:MaxConditions Caps:MaxConditionDepth Caps:MaxConditionSets
Caps:MaxConditionValues Caps:MaxAggregates Caps:MaxOrderFields Caps:MaxNavigationDepth Caps:MaxQueryCost
Caps:DefaultFieldCost Caps:MaxAuditEvents Caps:MinGroupSize Caps:SchemaDepth Caps:SchemaCycleLimit
Caps:MaxSchemaFields integers
```
```jsonc
{
"DynamicWhere": {
"Policies": {
"Tier": "Strict",
"DryRun": false,
"StoreFailure": "LastKnownGood",
"MaxSnapshotAge": "00:15:00",
"RefreshInterval": "00:00:30",
"Caps": { "MaxPageSize": 200, "SchemaDepth": 2, "MaxSchemaFields": 2000 }
}
}
}
```
Leave `Caps:MinGroupSize` out unless you mean it. Writing any value, even 5, sets `IsMinGroupSizeSet`.
### DwPolicyOptions
```
DwPolicyOptions sealed class; every setter throws InvalidOperationException once frozen
Tier DwTier Convenience
DryRun bool false
IncludeTraceInResult bool? null 3.1.0. null follows the tier: off under Strict, on under Convenience
AuditRefusals bool false 3.1.0. true also writes every refused guarded query to the audit
HashSalt string "" "" = none; null → ArgumentNullException; 1–15 chars → ArgumentException
TokenVault IDwTokenVault? null required only by MaskStrategy.Tokenize
Services IServiceProvider? null resolves [DwMutate] transformers
StoreFailure StoreFailureMode LastKnownGood
MaxSnapshotAge TimeSpan 00:15:00 <= 0 → ArgumentOutOfRangeException
RefreshInterval TimeSpan 00:00:30 <= 0 → ArgumentOutOfRangeException
Caps DwCaps get-only
Entities DwEntityCatalog get-only, empty
IsFrozen bool get-only
Freeze() -> void idempotent; also freezes Caps and Entities; Configure calls it
MinimumHashSaltLength const int = 16
DwTier Convenience=0 Strict=1
StoreFailureMode LastKnownGood=0 FailClosed=1 StaticOnly=2
```
- `Tier`:
- a denied Select or Order is dropped in `Convenience` and thrown in `Strict`;
- Where, Group and Aggregate denials throw in both tiers.
- `Strict` also refuses:
- `getQueryString: true`, with `QueryStringDenied` (14);
- a `Segment` condition on any field the caller may not Select, with `FieldDeniedForSegment` (6).
- `Strict` also discloses less (3.1.0):
- a guarded result carries no trace unless `IncludeTraceInResult` is true;
- outside dry run, a path that matches nothing on `T` is refused as a denied field is (section 17);
- every `FieldDeniedFor*` and `CapExceeded` refusal names no field: its `FieldPath` is `"*"`;
- inside a `Segment` every field refusal is `FieldDeniedForSegment`, whatever clause refused it;
- `MissingContextValue` names neither the scoped field nor the context key: `FieldPath` `"*"`, no `SourceOrigin`;
- outside dry run, `MaxQueryCost` is checked only after every field has passed its gate.
- `IncludeTraceInResult` (3.1.0) decides whether `FilterResult<T>.Policy`, `SummaryResult.Policy` and `SegmentResult<T>.Policy` carry the `PolicyTrace` from the guarded terminals.
- null, the default, follows the tier: off under `Strict`, on under `Convenience`. `true` or `false` overrides the tier in either direction.
- The trace names the fields a policy dropped, the attribute or rule that sealed each one, and every injected predicate: the detail `Strict` already refuses through `getQueryString`. An API that serializes a result sends it to the caller.
- The trace is still recorded on `PolicyQueryable<T>.LastTrace`, and audit events do not depend on the setting.
- `AuditRefusals` (3.1.0, default false) also writes every refusal a guarded query raises to the caller's audit buffer, drained to `IDwAuditSink` like `[DwAudit]` events (section 22).
- Off by default because it changes what reaches a sink, and the ASP.NET Core audit middleware warns on every request whose events find no sink.
- `DryRun` takes effect per query as `DwPolicyOptions.DryRun || DwPolicyContext.DryRun`.
- Decisions that would throw or drop are recorded in the trace and not enforced, and `PolicyTrace.DryRun` is true.
- The group floor is recorded but not applied.
- Still enforced in dry run:
- value transforms (results stay masked);
- the `MaxAuditEvents` refusal;
- `TransformRequiresMaterialization`;
- `PolicyRequired`;
- `PolicyContextNotPrepared` and `StoreUnavailable`.
- `HashSalt` keys `MaskStrategy.Hash`. Keep it stable for the life of a deployment: changing it changes every hashed value.
- `Services`: each `[DwMutate]` type is built through `Services.GetService(type)`, falling back to `Activator.CreateInstance(type)` (which needs a parameterless constructor).
- One instance per transformer type is cached for the process lifetime, so transformers must be stateless and thread-safe.
- A type that is not an `IValueTransformer`, or a null result → `InvalidOperationException`.
- `StoreFailure`, `MaxSnapshotAge` and `RefreshInterval` are read only by a `StorePolicyProvider`, from the options passed to its `CreateAsync`.
- Once a context's pinned snapshot is older than `MaxSnapshotAge`, its queries throw `StoreUnavailable` (17) in every mode except `StaticOnly` while the provider is degraded, which serves attributes alone and never reaches the ceiling.
- `Entities` lists the types that discovery APIs may describe. Nothing is describable until it is exposed.
### DwCaps
```
DwCaps sealed class; DwPolicyOptions.Caps. Setters: frozen → InvalidOperationException, below Min → ArgumentOutOfRangeException
Cap Default Min Counts Refusal
MaxPageSize 1000 1 Page.PageSize > cap; a request with no Page is bounded by DefaultPageSize
instead, if one is set CapExceeded (9)
DefaultPageSize 0 0 3.1.0. 0 = off. When set, a guarded query that sends no Page is given
PageNumber 1 and PageSize min(DefaultPageSize, MaxPageSize). A Page the
caller did send is never replaced. Negative → ArgumentOutOfRangeException refuses nothing
Applies to Filter, Summary and Segment, terminal and composable alike: the
composable Filter, FilterDynamic and Summary return the query already
paged, so page through the request's Page, not a chained Page(). Where,
Order, Select and Group take no page and are never given one.
MaxConditions 50 1 conditions at every nesting depth; Summary: ConditionGroup + Having;
Segment: all condition sets together CapExceeded (9)
MaxConditionDepth 10 1 3.1.0. How deep SubConditionGroups nest, root group = depth 1. Summary:
the deeper of ConditionGroup and Having; Segment: each set on its own CapExceeded (9)
MaxConditionSets 10 1 3.1.0. Segment.ConditionSets.Count, sets with no conditions included.
Each set adds a condition or subquery to one statement CapExceeded (9)
MaxConditionValues 1000 1 3.1.0. Values carried by the largest single condition: the where
clause, Having and every Segment set. An In / NotIn is one comparison
per value, so one condition could build a predicate of any size CapExceeded (9)
MaxAggregates 50 1 3.1.0. Summary GroupBy.AggregateBy.Count: Summary terminals and the
composable Group and Summary. The group floor's own count is not
counted CapExceeded (9)
MaxOrderFields 10 1 Orders.Count CapExceeded (9)
MaxNavigationDepth 4 1 dot segments of any path ("A.B.C.D" passes, 5 segments refused) in
conditions, Selects, Orders, GroupBy.Fields, AggregateBy.Field CapExceeded (9)
MaxQueryCost 1000 1 sum of cost over every field reference QueryCostExceeded (19)
DefaultFieldCost 1 0 cost of one reference to a field with no [DwCost] or rule weight, and
(3.1.0) of an aggregate with no Field, such as a Count
MaxAuditEvents 10000 1 undrained audit events one context may hold CapExceeded (9)
MinGroupSize 5 1 k-anonymity floor for guarded grouped summaries; 1 = off groups suppressed
SchemaDepth 2 1 default PolicySchemaRequest.Depth
SchemaCycleLimit 2 1 times one type may appear on one schema path
MaxSchemaFields 2000 1 fields in one PolicySchema; reaching it sets Truncated, never throws
IsMinGroupSizeSet bool, get-only; true once MinGroupSize has been assigned (code or configuration)
DefaultMinGroupSize const int = 5
```
- Caps apply only to guarded queries, simulations and schema building. Unguarded calls have no limits.
- The count caps — `MaxConditions`, `MaxConditionDepth`, `MaxConditionSets`, `MaxConditionValues`, `MaxAggregates`,
`MaxOrderFields` and `MaxPageSize` — are checked before any name is resolved (3.1.0), so an oversized request is
refused with `CapExceeded` even when it also names a field that does not exist; 3.0.0 resolved names first and
answered `ConditionMustHasValidFieldName`. `MaxNavigationDepth` needs canonical paths and is checked after them.
- A Segment is one statement: its sets are combined, ordered and paged in the database (section 6), so `MaxPageSize`
and `DefaultPageSize` bound what it reads as well as what it returns. `MaxConditionSets` bounds how many sets the
statement carries; a set with no conditions spends nothing from `MaxConditions` or `MaxConditionDepth`.
- `MaxConditionValues` compares the one condition carrying the most values, wherever it is. `MaxConditions` and the
cost budget see an `In` of any length as one condition and one field.
- Cost charges every reference, duplicates included, before gating, so a later-dropped field still costs.
- Filter: conditions, Orders, and the Selects the caller wrote.
- Summary: conditions, `GroupBy.Fields`, and every `AggregateBy` entry: its `Field`, or `DefaultFieldCost` for
one with no field, such as a `Count` (3.1.0; it was free). Having and Orders are not charged.
- Segment: every field it names.
- A projection the library synthesizes is free.
- The weight is the elected `[DwCost]` or rule weight, else `DefaultFieldCost`. Under `Strict`, a name that matches nothing costs `DefaultFieldCost` too.
- Where the total is checked depends on the tier (3.1.0). The Convenience tier and every dry run check it before gating. The Strict tier, outside dry run, checks it after every field has passed its gate: a field the caller may not use, weighted or not, is refused as denied first, exactly as a name that matches nothing is, so the budget cannot tell the two apart. An allowed weighted field still gets `QueryCostExceeded` in both tiers.
- Caps and cost count what the caller sent. Forced predicates, the group floor and a default order are added afterwards and count toward neither.
- A cap or cost refusal records a `Denied` decision before throwing.
- `FieldPath` is `"*"`, or, in the Convenience tier, the path for `MaxNavigationDepth`. Under `Strict` every `CapExceeded` refusal carries `"*"` (3.1.0); the trace keeps the path.
- `Reason` and `SourceOrigin` name the cap, e.g. `"MaxConditions cap (50), request had 51"`, `"MaxConditionValues cap (1000), request had 1001"`, `"MaxAggregates cap (50), request had 51"`.
- In dry run it is recorded and not thrown.
- `MaxAuditEvents` throws even in dry run, with `FieldPath` = the audited field (`"*"` under `Strict`).
- `MinGroupSize`:
- The effective floor is the largest of `Caps.MinGroupSize` and the `MinGroupSize` of the transform chain on any aggregated field.
- Above 1, a guarded `Summary` with a `GroupBy` gets a `Count` column `__dwGroupSize` plus `HAVING __dwGroupSize >= floor` in SQL.
- `TotalCount` and `PageCount` therefore count only surviving groups.
- `ToList` / `ToListAsync(Summary)` remove the column from the rows.
- A summary that itself uses `__dwGroupSize` (as an alias, in Having or in Orders) → `GroupTooSmall` (20).
- Every guarded grouping path applies it: `ToList` / `ToListAsync(Summary)` and the composable `Group` and `Summary`.
- `MinGroupSize = 1` switches the floor off with no warning. `IsMinGroupSizeSet` distinguishes that from a deployment that never set it.
### DwPolicyContext and DwSubject
```
DwPolicyContext sealed class
DwPolicyContext()
Subjects : IReadOnlyList<DwSubject> read-only view, insertion order
DryRun : bool { get; set; } dry run for this caller only
IsPrepared : bool { get; } 3.1.0. True once DwPolicy.PrepareAsync has run against it,
with or without a store. ApplyPolicy(ctx) refuses a
context where this is false
Purpose : string? { get; set; } matched by purpose-bound rules; copied into audit events
PendingAuditEvents : IReadOnlyList<DwAuditEvent> a copy of the undrained buffer
WithSubject(DwSubjectKind kind, string identity) -> DwPolicyContext mutates this instance, returns this
WithValue(string key, object? value) -> DwPolicyContext mutates this instance, returns this
Identities(DwSubjectKind kind) -> IEnumerable<string>
TryGetValue(string key, out object? value) -> bool
DwSubject sealed class : IEquatable<DwSubject>
DwSubject(DwSubjectKind kind, string identity)
Kind : DwSubjectKind
Identity : string trimmed, casing kept; "" for Global
Equals / GetHashCode Kind + Identity compared OrdinalIgnoreCase
ToString() "Global" | "<Kind>:<Identity>"
DwSubjectKind Global=0 Tenant=1 Role=2 User=3 Custom=4
```
- `WithSubject` and `WithValue` return the same instance. Nothing is copied, and a context is not immutable.
- `WithSubject`:
- a null or whitespace identity → `ArgumentException`, except for `Global`, whose identity is ignored;
- adding the same kind and identity again (case-insensitive) does nothing.
- `WithValue`:
- a blank key → `ArgumentException`;
- an existing key is replaced; keys are case-sensitive;
- values feed `[DwForceWhere(ContextValue = "key")]`, and an absent key → `MissingContextValue` (12).
- Use one context per request, built completely before `PrepareAsync`.
- A `User` subject added after `PrepareAsync` makes store-backed queries throw `PolicyContextNotPrepared`.
- Calling `PrepareAsync` again replaces what was pinned.
- A context serves the one snapshot pinned at `PrepareAsync`. Once it is older than `MaxSnapshotAge`, store-backed queries throw `StoreUnavailable`, so never cache contexts across requests.
- Queries may share one context concurrently, because the audit buffer is locked. Subjects and values are not synchronized: finish building before querying.
- There is no public correlation id or attachment API; `IsPrepared` (3.1.0) says only that `PrepareAsync` ran.
---
## 13. Guarded queries
### ApplyPolicy
```
PolicyExtensions static class, DynamicWhere.ex.Policies.Source
ApplyPolicy<T>(this IQueryable<T> query, DwPolicyContext context) -> PolicyQueryable<T> reads DwPolicy.Options + DwPolicy.Resolver
ApplyPolicy<T>(this IEnumerable<T> query, DwPolicyContext context) -> PolicyQueryable<T> in-memory, via AsQueryable()
ApplyPolicy<T>(this IQueryable<T> query, DwPolicyContext context,
DwPolicyOptions options, PolicyResolver resolver) -> PolicyQueryable<T> explicit posture; DwPolicy not read
where T : class; any null argument → ArgumentNullException
```
- `ApplyPolicy` builds the handle and resolves no policy. The two overloads that read `DwPolicy` refuse an unprepared context at the call, with `PolicyContextNotPrepared` (3.1.0), and write that refusal to the context's audit buffer when `DwPolicy.Options.AuditRefusals` is on; the four-argument overload checks nothing. Every other refusal comes from the method called on the handle.
- A resolver built with `new PolicyResolver(...)` for the four-argument overload does not include `AttributePolicyProvider`. Add `new AttributePolicyProvider()` yourself, or attributes are ignored.
### PolicyQueryable<T>
```
PolicyQueryable<T> where T : class sealed class; no public constructor
Terminal: sanitize, run, transform, set result.Policy (null under Strict unless IncludeTraceInResult)
ToList(Filter filter, bool getQueryString = false) -> FilterResult<T>
ToListAsync(Filter filter, bool getQueryString = false) -> Task<FilterResult<T>>
ToListDynamic(Filter filter, bool getQueryString = false) -> FilterResult<dynamic>
ToListAsyncDynamic(Filter filter, bool getQueryString = false) -> Task<FilterResult<dynamic>>
ToList(Summary summary, bool getQueryString = false) -> SummaryResult
ToListAsync(Summary summary, bool getQueryString = false) -> Task<SummaryResult>
ToListAsync(Segment segment) -> Task<SegmentResult<T>>
Terminal, cancellable (3.2.0): the same, with the token passed to the count and the read
ToListAsync(Filter filter, CancellationToken cancellationToken) -> Task<FilterResult<T>>
ToListAsync(Filter filter, bool getQueryString, CancellationToken cancellationToken) -> Task<FilterResult<T>>
ToListAsyncDynamic(Filter filter, CancellationToken cancellationToken) -> Task<FilterResult<dynamic>>
ToListAsyncDynamic(Filter filter, bool getQueryString, CancellationToken cancellationToken) -> Task<FilterResult<dynamic>>
ToListAsync(Summary summary, CancellationToken cancellationToken) -> Task<SummaryResult>
ToListAsync(Summary summary, bool getQueryString, CancellationToken cancellationToken) -> Task<SummaryResult>
ToListAsync(Segment segment, CancellationToken cancellationToken) -> Task<SegmentResult<T>>
Composable: sanitize one clause, apply the type's forced predicates, return a new handle
Select(List<string> fields) -> PolicyQueryable<T>
Where(Condition condition) -> PolicyQueryable<T>
Where(ConditionGroup group) -> PolicyQueryable<T>
Order(OrderBy order) -> PolicyQueryable<T>
Order(List<OrderBy> orders) -> PolicyQueryable<T>
Page(PageBy page) -> PolicyQueryable<T>
Filter(Filter filter) -> PolicyQueryable<T>
Composable, returning a query the caller materializes; each throws TransformRequiresMaterialization (16)
when any field of T is transformed for this caller
SelectDynamic(List<string> fields) -> IQueryable
Group(GroupBy groupBy) -> IQueryable
FilterDynamic(Filter filter) -> IQueryable
Summary(Summary summary) -> IQueryable
Other
AsUnguardedQueryable() -> IQueryable<T>
LastTrace : PolicyTrace?
```
How this differs from the unguarded surface:
- Composables return `PolicyQueryable<T>`, not `IQueryable<T>`. Finish with a terminal method or `AsUnguardedQueryable()`.
- There are no `IEnumerable<T>` overloads on the handle; use `ApplyPolicy(IEnumerable<T>)` instead.
- There is no synchronous Segment method, same as core.
Rules:
- Every guarded query runs over `AsNoTracking()`. Returned EF Core entities are detached, so a masked value is never
saved back. Through a provider that wraps EF Core's, as LinqKit's `AsExpandable` and DelegateDecompiler's
`Decompile` do, EF Core's extension would hand the query back still tracking, so the call is put into the query
itself (3.2.0). An in-memory source has no such copy: without `Selects` and with nothing denied, the rows returned are
the source objects themselves, transformed in place. When a projection is synthesized the rows are new, and they
hold no member of an object type, so the source objects are left as they were.
- Chaining keeps decisions: each composable returns a new handle, and the terminal's trace includes the earlier links' decisions.
- The terminal's trace is on `result.Policy` only when `DwPolicyOptions.IncludeTraceInResult` allows it: null follows the tier (off under `Strict`, on under `Convenience`), and `true` or `false` overrides it (3.1.0). `LastTrace` holds it whatever the setting.
- `getQueryString: true` under `Strict` → `QueryStringDenied` (14). This is checked before sanitizing; dry run records it instead.
- `TransformRequiresMaterialization` depends on the type, not on the fields named.
- It is thrown even in dry run, and `FieldPath` lists the transformed paths.
- Use `ToListDynamic` or `ToList(Summary)`, or leave deliberately with `AsUnguardedQueryable()`.
- `Where`, `Order` and `Page` never fail with `AllSelectsDenied`: no projection is synthesized for a single clause.
- `Group(GroupBy)` runs its sanitized summary through the summary pipeline, so forced predicates and the
`MinGroupSize` floor both apply, as they would on `ToList(Summary)`. Core `Group` takes no `Having`, which is
why it is not the path used.
- `Summary(Summary)` applies the floor's HAVING too. Neither returns the floor's own `__dwGroupSize` column —
both project it back out — and both keep canonical column names, because alias renaming happens on
materialized rows.
- `AsUnguardedQueryable()` returns the handle's source as a plain `IQueryable<T>`.
- It keeps what earlier composable calls applied: forced predicates, clauses, `AsNoTracking`.
- It skips everything after: no gating, no transforms (values come back unmasked), no trace.
- On a fresh handle it is the original, tracked source with no predicates.
- Calling a DynamicWhere extension on it for a `RequirePolicy` type → `PolicyRequired`.
- `LastTrace` is set on the handle the method was called on, not on the handle it returns.
- It holds the latest call that got past sanitizing, or a `QueryStringDenied` refusal.
- Refusals throw `PolicyException` (a `LogicException`) from the handle method.
- With `DwPolicyOptions.AuditRefusals` on, the terminal and composable methods also write the refusal to the context's audit buffer on its way out, once, without changing or catching it (section 22).
### Default order — `[DwEntity(DefaultOrder = ...)]` (3.1.0)
```csharp
[DwEntity(DefaultOrder = "CreatedAt desc, Id")] // DynamicWhere.ex.Policies.Attributes
public class Ticket { … }
```
- The order a guarded query takes when its caller sends none. Comma-separated entries, each a field path
(navigations allowed) optionally followed by `asc` or `desc` in any letter case; ascending when neither. A blank
entry, such as the one a trailing comma leaves, is ignored, and a field named twice is used once.
- Applied only under `ApplyPolicy`, when `Orders` is null or empty: `ToList`, `ToListAsync`, `ToListDynamic` and
`ToListAsyncDynamic` with a `Filter`; `ToListAsync(Segment)`; the composable `Filter` and `FilterDynamic`; and the
composable `Page` on a source nothing has ordered and whose projection hides no default field.
- Never applied:
- by an unguarded call. The core methods of section 6 on a plain `IQueryable<T>` or `IEnumerable<T>`, `Page`
included, never read the attribute and order only as their caller asks, exactly as in 3.0; so does a DynamicWhere
method called on what `AsUnguardedQueryable()` returns;
- when the caller sends orders: the default is not appended as a tiebreak;
- to a source already ordered, before it was guarded (`db.Tickets.OrderBy(t => t.Title).ApplyPolicy(ctx)`) or by
a composed `Order` earlier in the chain — even one whose every order the policy dropped, because the caller
still sent orders. A composed `Filter` that sent orders counts the same way (3.2.0). Only the query's expression
is read, so a sequence sorted in memory before `ApplyPolicy(IEnumerable<T>)` does not count as ordered: send
`Orders` for it;
- to a source whose projection could hide a default field. Only the outermost `Select` of the chain counts, because
it makes the rows the default orders. Since 3.2.0 it hides nothing when it builds T itself in an object
initializer, `Select(t => new Row { Code = t.Code, … })`, and assigns every field the default names, at every
level of a nested path (`"Owner.Name"` needs `Owner = new OwnerRow { Name = … }`), a column. On EF Core a column
is a member the model maps on the entity the `Select` reads, read directly, through reference navigations
(`t.Owner.Name`) or through `EF.Property` (a shadow property included); the default then applies, because EF
Core translates an order by a column the projection assigned. In memory any assigned field is ordered by. A
value the projection computes, by a method (`Regex.Replace`, `ToUpper`, the application's own) or an operator
(`t.First + " " + t.Last`), a member the model does not map, a constructor with arguments, a default field the
initializer does not assign, or a nested path through anything but an initializer leaves the query in its own
order, as every projection did in 3.1.0: ordering by it could fail where the unguarded query ran;
- after a projection composed on the handle: the guarded `Select`, or a guarded `Filter` whose `Selects` is set,
leaves the rest of the chain unordered even when it keeps every default field, so
`guarded.Select(["Id", "Title"]).Page(page)` pages as it did in 3.0.0, unordered. The default is for the rows the
caller's source makes;
- to a `Summary`, or by the composable `Where`, `Select` and `Order`.
- A type that declares no `DefaultOrder` is never ordered by the library, guarded or not; only a caller's own
orders apply. End a default with a unique field, such as the key, or rows sharing the leading values can still
change places between pages.
- `[DwEntity]` allows one attribute per type and is inherited the .NET way: a `[DwEntity]` on a derived type replaces
its base type's instead of merging with it. `[DwEntity(RequirePolicy = true)]` on a type whose base declares
`DefaultOrder` has no default order, and the reverse drops `RequirePolicy`. Repeat both on the derived type.
- An entry naming a field the type does not have, one that is not a field and a direction (`"Id sideways"`), one
the core refuses to order by, a path ending on a collection of entities (`"Tags"`), or one whose name the parser
keeps for itself (`"Null"`, section 5) is skipped, never refused, and `PolicyModelValidator` reports all four
(section 20). A path through a collection to a value (`"Tags.Value"`) is
kept and sorted as section 6 sorts it, by the smallest value ascending or the largest descending.
- The sanitizer applies it for this caller, after the caller's own orders are gated:
- a default field this caller may not order by is left out, never refused, and recorded as a `Dropped` decision
on `Order` whose reason starts `left out of the default order`. Ordering by it would rank rows by a value the
caller may not see;
- in a `Segment`, a default field this caller may not use in a segment is left out too, recorded the same way,
because a segment refuses that field in any clause. A `Filter` still takes it;
- a dry run keeps the field and still records the decision;
- a field the default keeps is a use of that field. One audited for `Order`, by `[DwAudit]` or a rule, is recorded
as a use, `Effect` `Allow`, each time a guarded query orders by it, as a caller's own order is. A field the
default leaves out is not recorded: the query does not order by it, and the caller never named it. A dry run
keeps the field, so it records it, with its `Order` effect (`Deny` for a field the caller may not order by) and
`DryRun` true (section 22);
- a caller whose sent orders were all dropped (Convenience) gets no default in their place;
- it is added after the caps and after the cost is counted, so it counts toward no cap and costs nothing.
### [DwEntity(RequirePolicy = true)] enforcement
- All 28 public methods of `DynamicWhere.ex.Source.Extension` (the `IQueryable<T>` and `IEnumerable<T>` overloads) run the guard first.
- They throw `PolicyException` with `ErrorCode = PolicyRequired` (10) when `T` requires a policy and the call is not running inside a `PolicyQueryable<T>` method.
- Composables such as `Select` and `Where` throw when called, not when enumerated.
- The exception carries:
- `FieldPath` = `typeof(T).Name`;
- `Feature` = `None`;
- `Tier` = `Strict` (whatever is configured);
- `SourceOrigin` = `"DwEntityAttribute(RequirePolicy = true)"`.
- Neither dry run nor `DwPolicy.IsConfigured` affects it.
- It checks `T`, the query's element type. The attribute is inherited by subclasses, and the answer is cached per type for the process lifetime.
- A subclass that declares a `[DwEntity]` of its own, for a `DefaultOrder` say, replaces the inherited one, and its `RequirePolicy` is false unless it says `true` again.
- Not guarded:
- plain EF Core or LINQ on the type (`db.Employees.ToListAsync()`, `.Where(e => ...)`);
- anything done with `AsUnguardedQueryable()` that avoids DynamicWhere extension methods.
---
## 14. Policy attributes
Namespace `DynamicWhere.ex.Policies.Attributes`; `[DwX]` is class `DwXAttribute`. There are 22
attributes plus the abstract base `DwPolicyAttribute`.
```
AttributeUsage (every attribute: Inherited = true)
DwEntity Class AllowMultiple = false
DwDeny DwDenied DwNoWhere DwNoSelect DwNoOrder DwNoGroup
DwNoAggregate DwOperators DwForceWhere Property|Field AllowMultiple = true
every other attribute Property|Field AllowMultiple = false
```
- Only public instance properties are read, including properties reached through navigations and
collection elements, up to paths of 4 segments. An attribute on a field compiles and does nothing.
- Every attribute except `DwEntity` derives from `DwPolicyAttribute` and has `Overridable:bool = false`.
False places it at `PolicyLevel.SealedAttribute`; true places it at `PolicyLevel.OverridableAttribute`.
- A type's attributes are read once and cached. A malformed one throws `ArgumentException` on every
guarded query of that type, and of every type that navigates to it.
```
Access control
[DwEntity] RequirePolicy:bool = false DefaultOrder:string? = null (3.1.0)
[DwDeny(PolicyFeature features)] Features:PolicyFeature
[DwDenied] DwDeny(PolicyFeature.All)
[DwNoWhere] DwDeny(PolicyFeature.Where)
[DwNoSelect] DwDeny(PolicyFeature.Select)
[DwNoOrder] DwDeny(PolicyFeature.Order)
[DwNoGroup] DwDeny(PolicyFeature.Group)
[DwNoAggregate] DwDeny(PolicyFeature.Aggregate)
[DwOperators] Allow:Operator[]? = null Deny:Operator[]? = null
Resolve() -> IReadOnlyList<Operator>
Injection
[DwAlias(string name)] Name:string
[DwForceWhere(Operator op)] Operator:Operator Value:string? = null
ContextValue:string? = null AllowNull:bool = false (3.1.0)
[DwRequireWhere] Operators:Operator[]? = null Resolve() -> IReadOnlyList<Operator>
static DefaultOperators = { Equal, IEqual, In, IIn }
Transformation (each also has AllowAggregate:bool = false MinGroupSize:int = 0)
[DwMask(MaskStrategy strategy)] Strategy KeepStart:int = 0 KeepEnd:int = 0
MaskChar:char = '*' PreserveLength:bool = true
Pattern:string? = null Replacement:string? = null
Text:string? = null TokenScope:string? = null
[DwMutate(Type transformer)] Transformer:Type
[DwDefault] [DwDefault(string value)] Value:string? HasValue:bool (true only via the string ctor)
[DwGeneralize(GeneralizeMode mode)] Mode Step:int = 0 Part:DatePart = DatePart.Year
Decimals:int = 0
[DwTruncate(int length)] Length:int Ellipsis:string? = null
[DwFormat(string format)] Format:string
Discovery, budget, audit
[DwDescribe] Label:string? Description:string? Group:string? Order:int
[DwAllowedValues(params string[] values)] Values:string[]
[DwCost(int weight)] Weight:int
[DwAudit] [DwAudit(PolicyFeature features)] Features:PolicyFeature (no-arg ctor = PolicyFeature.All)
```
No named attribute refuses `Segment`: write `[DwDeny(PolicyFeature.Segment)]`.
### DwEntity
- `RequirePolicy = true`: any DynamicWhere.ex extension method on this `T` throws `PolicyException`
`PolicyRequired` when called outside a guarded call. That covers `Select`, `SelectDynamic`, `Where`,
`Order`, `Page`, `Group`, `Filter`, `FilterDynamic`, `Summary` and every `ToList*`, on `IQueryable<T>`
and `IEnumerable<T>`. The exception's `FieldPath` is the type name and `Tier` is `Strict`.
- It is checked whether or not `DwPolicy.Configure` ran. Plain LINQ or EF on the `DbSet` is not
intercepted.
- `DefaultOrder` (3.1.0), e.g. `[DwEntity(DefaultOrder = "CreatedAt desc, Id")]`: the order a guarded query
takes when its caller sends none, less the fields this caller may not order by. Unguarded calls ignore it.
Entry syntax, entry points and exclusions are in section 13; `ValidateModel` checks it (section 20).
- A derived type reads its own `[DwEntity]` when it has one, and that attribute replaces the base type's whole: set
`RequirePolicy` and `DefaultOrder` again on it.
### DwDeny family
- Each attribute refuses its features on that exact path. Several on one member all apply.
- A denial on a navigation (`Contact`) covers only `Contact`. `Contact.Email` is a separate field.
- `[DwDeny(PolicyFeature.None)]` refuses nothing, and nothing reports it.
- `[DwNoSelect]` leaves the field filterable, sortable, groupable and aggregatable, so it can still be
counted.
- One on another declaration of the member applies to the path too (3.2.0): on the interface member a class, or a
loaded subtype of it, implements with the member; on an override of either accessor a loaded subtype declares; on
a public member a loaded subtype hides with `new`; and on the implementation a loaded type gives an interface
member, explicit, inherited from a base class or declared by an open generic class, through that interface or an
instantiation variance lets stand for it (`IFeed<VisaCard>` for a member typed `IFeed<Card>`, with `out T`). A row read through the base type or
the interface is still that subtype, and its member returns what the subtype's declaration returns, so the denial
holds for the path on every row, Where, Order, Group and Select alike. A member hidden with `new` counts because
whether it reads the member it hides cannot be told from outside, and a row serialized as its own type writes it
under the same name. Only the deny family is read this way; aliases, transforms and the other attributes are read
from the declaration walked.
### DwOperators
- `Resolve()` returns `Allow` (every `Operator` when null) minus `Deny`. An operator in both lists is
refused. `Allow = new Operator[0]` permits nothing, so the field cannot be filtered at all.
- Every restriction matching the field is intersected: repeated attributes, rules at any level, and `"*"`
rules. A rule can only narrow the set, so `Overridable` changes nothing.
- It is checked on every WHERE condition at any depth and in every Segment set. It is also checked on a
`Having` condition that reaches the field through a group key or an aggregate alias.
- A failure throws `OperatorNotAllowed` in both tiers. If the field is also denied for Where,
`FieldDeniedForWhere` is raised instead.
- It is never applied to forced predicates.
### DwAlias
- `Name` is trimmed. A blank, dotted (`a.b`) or `"*"` name throws `ArgumentException`.
- Two members of one type with the same alias (case-insensitive) is a `ValidateModel` error.
- The alias is accepted wherever a path is: condition `Field`, `Selects`, `Orders`, `GroupBy.Fields` and
`AggregateBy.Field`. It is also accepted in `Having` and summary `Orders` names that refer to an aliased
group key.
- Matching is case-insensitive, and the real path still works.
- A name that could mean two fields throws `AmbiguousFieldName`, in both tiers and in dry run. That
happens when an alias equals another real path or another alias.
- Exception: one member reached both at the root and through navigations (`Code`, `Manager.Code`)
resolves to the root.
- `PolicyException.FieldPath` carries the name the caller wrote. The trace records the canonical path,
with the alias in the reason. Under the Strict tier a `FieldDeniedFor*` or `CapExceeded` refusal names no
field at all, alias or path: its `FieldPath` is `"*"` (3.1.0, section 17).
- Output renaming happens only in `ToListDynamic`, `ToListAsyncDynamic` and
`ToList`/`ToListAsync(Summary)`.
- A row holding an aliased column is rebuilt as an `ExpandoObject` with that column under the alias.
- A nested path's column is matched by its path without dots.
- Typed `FilterResult<T>` and `SegmentResult<T>` rows keep their member names.
- An alias that stands for more than one path is not renamed.
- An alias is not applied on a type reached from itself (`Employee.Manager.Code`).
### DwForceWhere
- Set exactly one of `Value` or `ContextValue`. `Operator.IsNull` and `Operator.IsNotNull` take neither.
Any other combination throws `ArgumentException` on every guarded query of the type. Since 3.1.0
`ValidateModel` reports it at startup, and so it does an unsupported member type and a misused
`AllowNull`.
- `DataType` comes from the member's CLR type:
- enum → `Enum`
- `string`, `char` → `Text`
- `Guid` → `Guid`
- `bool` → `Boolean`
- `DateOnly` → `Date`
- `DateTime`, `DateTimeOffset` → `DateTime`
- any numeric type → `Number`
- any other type throws `ArgumentException`.
- `Value` is the injected condition's only value, parsed by the query pipeline.
- `ContextValue` is a key looked up with `DwPolicyContext.TryGetValue`. Keys are set by `WithValue` and
are case-sensitive.
- A missing or null context value throws `MissingContextValue` in both tiers. In dry run it is recorded
and nothing is injected.
- Convenience: `FieldPath` is the scoped field and `SourceOrigin` names it and the context key.
- Strict (3.1.0): `FieldPath` is `"*"` and `SourceOrigin` is null, so the message names neither the scope's
column nor the key it reads, which together describe how rows are partitioned. The trace keeps both, and an
`AuditRefusals` event records the scoped field's canonical path.
- Each predicate carries one value, or none for a null check, so `Between` and `NotBetween` cannot be
forced.
- `AllowNull = true` (3.1.0) lets a row whose member is null through as well: the injected term is
`(field op value OR field IS NULL)`. It is for a row that belongs to one tenant or to none, such as a
system role no institution owns. Two forced predicates on one member are joined by `And`, so no
combination of them can say "or null".
- `[DwForceWhere(Operator.Equal, ContextValue = "TenantId", AllowNull = true)] public int? InstitutionId`
- It works with every operator that takes a value. With `Operator.IsNull` or `Operator.IsNotNull`, or on
a member that can never be null (a non-nullable value type), it throws `ArgumentException` when the
type's policy is resolved, and `ValidateModel` reports it.
- A `ContextValue` is still required: a missing or null one throws `MissingContextValue`. The rows that
pass widen; the caller's own scope does not.
- The term is a disjunction, so it does not satisfy a `[DwRequireWhere]` on the same member.
- Dry run injects nothing, as for every forced predicate.
- The predicate is added after gating. It is never checked against the caller's own policy and never
counts toward caps or cost. Shape:
`ConditionGroup { Sort = 0, Connector = And, Conditions = forced (Sort 0,1,…), SubConditionGroups =
[caller's group, Sort reset to 0, then one Or group per AllowNull predicate (Sort 1,2,…)] }`. With no
caller group, only the forced terms are sent. The result reads `(A OR B) AND TenantId = 5`, or with
`AllowNull` `(A OR B) AND (InstitutionId = 5 OR InstitutionId IS NULL)`: a caller's `Or` never merges
with a forced term.
- It applies to `Filter`, to `Summary.ConditionGroup` (not `Having`), to every Segment condition set, and
to the guarded composable methods.
- A Segment with no sets gets one set holding the scope.
- The trace records `PolicyAction.Injected` on `Where`, with reason `forced predicate (Equal)`, or
`forced predicate (Equal, or null)` for a term that admits null (the operator name varies).
- Predicates are collected and ANDed, never elected: repeated attributes, rules at every level, `"*"`
rules. No rule can remove one, so `Overridable` changes nothing.
- A predicate declared on a navigated type is injected through the navigation, up to 4 segments, except
onto a type reached from itself. For example, a query on `Order` injects `Buyer.TenantId`; through a
collection the condition becomes `Any`.
### DwRequireWhere
- `Operators = null` means `DefaultOperators` (Equal, IEqual, In, IIn). An empty array means nothing
satisfies the requirement, so every query on the type is refused.
- It is satisfied only by a WHERE condition on the field whose operator is in the set and that is in a
narrowing position. The condition's group and every ancestor group must use `Connector.And` or hold at
most one child (conditions plus subgroups).
- `Status = A OR TenantId = 5` does not satisfy it.
- A `Having` condition never does.
- It is checked after injection, so a `[DwForceWhere]` on the same field satisfies it when its operator
is in the set. Dry run injects nothing. A forced predicate with `AllowNull = true` never satisfies it:
its term sits in an `Or` group, which is not a narrowing position, so the caller must still filter on
the field (3.1.0).
- A missing filter throws `RequiredFilterMissing` in both tiers. `FieldPath` is the field's alias when it
has one.
- In a Segment, every condition set must satisfy it.
- It is checked on every guarded call, so a lone `.Order(...)` or `.Page(...)` on the handle also throws.
- It follows navigations the same way `[DwForceWhere]` does: a query on `Order` can require
`Buyer.Division`.
- It is elected: a rule cannot lift a sealed requirement but may add one where none exists. A `"*"` rule
cannot carry one.
### DwDescribe, DwAllowedValues
- Both are schema metadata and decide nothing. `[DwAllowedValues]` is not enforced: a filter on an
unlisted value is allowed.
- `Label`, `Description`, `Group`, `Order` and `AllowedValues` are each elected separately. A rule that
sets only a label keeps the attribute's other values.
- An unset `Order` reads as 0 but counts as not set.
- `[DwDescribe]` with nothing set, or `[DwAllowedValues]` with no values, is a `ValidateModel` error. A
blank text value or blank list entry throws `ArgumentException`.
- A `"*"` rule cannot carry either.
### DwCost
- `Weight = 0` means the field is free. A negative weight is a `ValidateModel` error and throws
`ArgumentOutOfRangeException`.
- Every reference the caller writes is charged before gating, so a denied field costs the same as an
allowed one.
- Filter: each condition at any depth, and each `Orders` and `Selects` entry.
- Summary: `ConditionGroup` conditions, `GroupBy.Fields`, and each `AggregateBy` entry — its `Field`, or
`Caps.DefaultFieldCost` for an aggregate with no field, such as a `Count` (3.1.0; before, it was free).
- Segment: every field in every clause.
- Not charged: a synthesized projection, `Having`, summary `Orders`, forced predicates and the group-size
count.
- An unweighted field costs `Caps.DefaultFieldCost` (default 1, may be 0).
- A total above `Caps.MaxQueryCost` (default 1000) throws `QueryCostExceeded` in both tiers. In dry run
it is only recorded.
- The Convenience tier checks the total before gating. The Strict tier checks it after every field gate
(3.1.0), so a weighted field the caller may not use is refused as denied, exactly as a name that matches
nothing is, before its weight could set the two apart.
- The weight is elected: the top-ranked one wins, and among fragments tied at that rank the largest wins.
A `"*"` rule may set a weight for every field.
### DwAudit
- `PolicyFeature.None`, or a value with an undefined bit, is a `ValidateModel` error and throws
`ArgumentException`.
- One `DwAuditEvent` is recorded per use of an audited feature on a field the request names, whether the
use is allowed or refused. A field named twice is recorded twice. Events are buffered on the context
until drained.
- A field the type's `DefaultOrder` adds is recorded too (3.1.0): audited for `Order`, it is an `Order` use,
`Effect` `Allow`, each time a guarded query orders by it (section 13).
- Not recorded:
- columns returned because the request had no `Selects`
- a `DefaultOrder` field left out for this caller, outside a dry run
- forced predicates
- output transforms
- If the context already holds `Caps.MaxAuditEvents` (default 10000) events, the query throws
`CapExceeded`, in both tiers and in dry run.
- A refusal becomes an event of its own, with an `ErrorCode`, only when `DwPolicyOptions.AuditRefusals` is
on, and then for every refused guarded query, audited field or not (3.1.0, section 22).
- It is elected: the top-ranked audit wins outright, and fragments tied with it are unioned. A rule can
neither widen nor narrow a sealed `[DwAudit]`. A `"*"` rule may audit every field.
---
## 15. Policy enums
Namespace `DynamicWhere.ex.Policies.Enums`, except `TransformKind`, which is in
`DynamicWhere.ex.Policies.DTOs`. Stored rules name enum members by name, never by number.
```
PolicyFeature [Flags] None=0 Where=1 Select=2 Order=4 Group=8 Aggregate=16 Segment=32 All=63
PolicyLevel SealedAttribute=1 DynamicUser=2 DynamicRole=3 DynamicTenant=4 DynamicGlobal=5
OverridableAttribute=6
PolicyEffect Allow=0 Mask=1 Deny=2
PolicyAction Allowed=0 Denied=1 Dropped=2 Masked=3 Injected=4 Mutated=5 Defaulted=6 Generalized=7
DwTier Convenience=0 Strict=1
DwSubjectKind Global=0 Tenant=1 Role=2 User=3 Custom=4
MaskStrategy Full=0 Partial=1 Email=2 Phone=3 Regex=4 Fixed=5 Hash=6 Null=7 Tokenize=8
GeneralizeMode Round=0 Bucket=1 DatePart=2 Truncate=3
DatePart Year=0 Quarter=1 Month=2 Day=3
TransformKind Mutate=0 Generalize=1 Format=2 Mask=3 Truncate=4 Default=5
StoreFailureMode LastKnownGood=0 FailClosed=1 StaticOnly=2
```
```
Where / Select / Order conditions (plus Having through a key or alias) / Selects / Orders
Group / Aggregate GroupBy.Fields / AggregateBy.Field
Segment any use inside a Segment
PolicyFeature.None fragment that only carries something: alias, operators, predicate,
requirement, facts
PolicyEffect.Mask allowed and transformed on output; emitted on Select by transform attributes
PolicyAction.Allowed untouched; also recorded when an alias renames an output column
PolicyAction.Denied refused: thrown, or recorded in dry run
PolicyAction.Dropped removed quietly: Convenience Order/Select, synthesized projection, group floor
PolicyAction.Injected forced predicate added
Masked/Mutated/ one per transformed path; Defaulted > Mutated > Masked > Generalized;
Defaulted/Generalized a chain of only Format/Truncate records Masked
DwTier.Convenience the default; refused Order and Select entries are dropped
DwTier.Strict refused Order/Select throw; getQueryString and Segment filters on
select-denied fields are refused; results carry no trace unless
IncludeTraceInResult; an unknown path is refused as a denied field is,
and field and cap refusals name no field (3.1.0)
DwSubjectKind rule level: User→DynamicUser, Role→DynamicRole, Tenant and Custom→DynamicTenant,
Global→DynamicGlobal (takes no key)
StoreFailureMode LastKnownGood (default) serves the last snapshot up to MaxSnapshotAge;
FailClosed refuses guarded queries (StoreUnavailable); StaticOnly enforces
attributes alone
```
---
## 16. Precedence
```
Level PolicyLevel Comes from
1 SealedAttribute attribute with Overridable = false (the default)
2 DynamicUser rule for DwSubjectKind.User
3 DynamicRole rule for DwSubjectKind.Role
4 DynamicTenant rule for DwSubjectKind.Tenant or DwSubjectKind.Custom
5 DynamicGlobal rule for DwSubjectKind.Global
6 OverridableAttribute attribute with Overridable = true
```
`DwPolicy.Configure` always adds `AttributePolicyProvider`. No rule can reach level 1.
Resolution for one field path and one feature (`Where`, `Select`, `Order`, `Group`, `Aggregate`,
`Segment`):
1. A fragment matches when its path equals the field path, compared case-insensitively after trimming
each segment and dropping empty ones. A path of `"*"` also matches: it means every field of the entity,
nested paths included. `"Contact.*"` is a literal path, not a wildcard.
2. Among matching fragments whose `Features` include the feature, the winner is decided by, in order:
- the lowest level;
- an exact path over `"*"`;
- the higher `Priority` (rules only; attributes are 0);
- the stronger `PolicyEffect`: `Deny` > `Mask` > `Allow`.
A complete tie keeps the first fragment found, which has the same effect anyway.
3. Weaker levels are discarded, not merged. When nothing matches, the feature is allowed.
4. Afterwards, if the field has a transform stage and any stage lacks `AllowAggregate`, `Aggregate` is set
to `Deny`, overriding whatever won.
```
Fragment from Enters the feature contest as Also carries
DwDeny family Deny on Features -
transform attribute Mask on Select one transform stage
every other attribute PolicyFeature.None (no contest) operators / predicate / alias / requirement / fact
```
```
Carrier Combined how
AllowedOperators intersected across every match at every level (empty = none)
forced predicates all kept, ANDed
Alias, RequiredOperators elected by the ranking above
each TransformKind stage elected separately per stage kind
Label Description Group Order AllowedValues elected separately per fact
CostWeight elected; ties at the winning rank take the largest
AuditedFeatures elected; ties at the winning rank are unioned
```
- Sealed means no rule can win anything a sealed attribute decides:
- its Deny;
- the Mask on Select that a transform attribute adds, so no rule can deny Select on a field with a
sealed transform;
- its stage kind, alias, requirement, fact, weight and audit.
A rule can still decide features the attribute does not touch, and can add stages of other kinds.
- An `Overridable` attribute loses to any rule, at any dynamic level, that decides the same feature or
carries the same stage kind or fact.
- `Overridable` does nothing on `[DwOperators]`, which are intersected, or `[DwForceWhere]`, which are
collected.
- A rule never removes a transform stage. It can only win that stage kind with a stage of its own, and a
stored rule cannot carry `Mutate`. An Allow rule on Select leaves the mask running.
- Level is compared first. A user rule beats a role rule. A `"*"` rule at a stronger level beats an exact
rule at a weaker one.
- Within one level, an exact-path Allow beats a `"*"` Deny, and a higher-Priority Allow beats a Deny.
Conflicting role rules land on the stricter effect only when they tie on specificity and Priority.
- If a field is both denied and transformed at the same level, Deny wins: the field is dropped or refused,
not masked.
- Every store and the admin endpoint call `SealedFields.Refuse` before saving a rule. It rejects a rule
whose path and features overlap a sealed attribute, provided the store can resolve the entity type.
Resolution enforces the ceiling regardless.
The resolver, fragment and provider types are listed in section 21.
---
## 17. Enforcement by tier and dry run
The count caps are checked first, before any name is resolved (3.1.0). Names are then resolved to canonical paths,
`MaxNavigationDepth` is checked, and the cost budget is checked before field policies — except under `Strict` outside
dry run, where it is checked after them (3.1.0). Nothing is clamped.
```
Request Convenience Strict Error code
WHERE on a field denied for Where (any depth, any set) throw throw FieldDeniedForWhere
WHERE with an operator the field does not allow throw throw OperatorNotAllowed
HAVING through a key or alias of such a field or operator throw throw one of the two above
ORDER BY a field denied for Order (also a summary key/alias) drop throw FieldDeniedForOrder
SELECT a field denied for Select drop throw FieldDeniedForSelect
SELECT a navigation with a denied field beneath it allowed leaves throw FieldDeniedForSelect
SELECT a navigation that cannot be narrowed around a denied
field: its key a.Id is denied, or a path through it is one
the core cannot project (3.2.0) throw throw FieldDeniedForSelect
SELECT a.b when the key a.Id is denied throw throw FieldDeniedForSelect
SELECT a member that can carry a denied field no path names:
past four segments, in a framework generic, on a subtype,
or unasked under a "*" deny (3.2.0) allowed leaves, throw FieldDeniedForSelect
or throw where it
cannot be narrowed
every requested SELECT dropped throw - AllSelectsDenied
no Selects while a denied field can reach the result (3.2.0) allowed members allowed members
GROUP BY a denied field throw throw FieldDeniedForGroup
AGGREGATE a denied field, or a transformed field lacking
AllowAggregate on a stage throw throw FieldDeniedForAggregate
field denied for Segment used anywhere in a Segment throw throw FieldDeniedForSegment
Segment condition on a field denied for Select allowed throw FieldDeniedForSegment
any other field refusal inside a Segment (3.1.0) the rows above throw Strict: FieldDeniedForSegment
[DwRequireWhere] not satisfied throw throw RequiredFilterMissing
ContextValue missing or null throw throw MissingContextValue
MaxConditions / MaxConditionDepth / MaxConditionSets /
MaxConditionValues / MaxAggregates (3.1.0) /
MaxOrderFields / MaxPageSize /
MaxNavigationDepth exceeded throw throw CapExceeded
total cost above MaxQueryCost throw throw QueryCostExceeded
getQueryString: true SQL returned throw QueryStringDenied
a path that matches nothing on T (3.1.0) throw throw Convenience: ConditionMustHasValidFieldName
Strict: that clause's FieldDeniedFor* code
DefaultOrder field the caller may not order by (3.1.0) left out left out
DefaultOrder field denied for Segment, in a Segment (3.1.0) left out left out
```
- "drop" removes the entry and records `Dropped`. A Strict throw records `Denied`.
- The guarded composable `Order(...)` and `Select(...)` drop and throw the same way.
- "allowed leaves": when `Selects` names a navigation with a denied field beneath it, it is replaced by the allowed
leaf paths beneath it. A navigation with nothing denied beneath it is kept as written, unless a transform beneath it
lands on a property with no setter, which the outbound walk could not write back: it is then narrowed around that
property (3.2.0).
- The fields beneath are read the way the attribute walker reads them (3.2.0): through any collection type, so a
member typed `IReadOnlyList<T>` or an application's own collection no longer hides its denied fields, and no
deeper than the walker's four segments. The providers' fragments are asked too, so a denied property with no
setter and a rule on a path reached through a cycle count (3.2.0).
- A named member can also carry a field denied for Select that no path names (3.2.0): deeper than four segments,
inside a framework generic such as `Dictionary<string, T>`, or declared by a subtype of the member's type (a
derived entity, a subclass, an interface's implementation). What it can carry is read from the source. On an
entity it is read from the EF Core model, so only what loads counts: the navigation's columns, a converted one
included, its owned chain at any depth, the navigations beneath it that an include, an automatic include or a
lazy loader fills, and each member the model does not map, read as its type, since its getter can hand out what
EF Core loaded; for its type and every type the model derives from it. On a projected row it is the type the
initializer constructs the member as, when it says (`Contact = new ContactRow { … }`), and otherwise the
member's type and every loaded subtype of it, as on a row in memory. Under a policy with a `"*"` deny, a path
the walk never asks about (past four segments, with no setter, or on a subtype) is a denied one unless the
policy names it; around a cycle, where the paths never end, it always is.
- Such a member is refused with `FieldDeniedForSelect` under Strict. Under Convenience it is narrowed to the
allowed leaves where the core can narrow the path's first member, which builds the declared type and so drops a
subtype's fields; a path that names a framework generic itself narrows to nothing and is dropped. Where the core
cannot narrow it (a column, complex or JSON member at the top of T, or a member of a row in memory) it is
refused in both tiers.
- A narrowing that cannot be built as gated is refused with `FieldDeniedForSelect`, in both tiers (3.2.0). The
core's typed projection adds the key (`Id`) of every nested node it builds, so a narrowing whose nodes carry a
denied key would return it; a path through a collection the core does not unwrap fails its validation; and a
column, complex property or JSON-stored member, a member of a row in memory, or one a projection builds some way
the core cannot narrow, cannot be narrowed at all.
- A navigation named through another, `Main.Lead`, gates the key of every node it passes through, which the
builder adds, as a dotted path to a value always did (3.2.0). A denied one refuses the projection.
- "allowed members" (3.2.0; "allowed scalars" before) applies when `Selects` is null or empty and a field is denied
for Select where its value can reach the result. It runs for a whole-`Filter` terminal and for a Segment, not for a
single-clause composable call.
- A denial at the top of T always counts, whatever the member holds: a scalar, a blob, a list, an owned object, a
JSON column (3.2.0; 3.1.0 asked only about simple members, so a denied `byte[]` or owned member came back).
- A denial beneath a member counts when its value can reach the result. A rule may spell its path in any letter
case.
- On an entity: beneath a column, an owned or complex member, or a navigation something loads. That is an
`Include` or `ThenInclude` on the query, an automatic include, or a lazy loader: proxies, an injected
`ILazyLoader`, a loader delegate or `ILazyLoader` the constructor takes, kept in a field or any property, the
asynchronous loader delegate EF Core 7 added, or an injected `DbContext`, any of which fills a navigation
after the query. Every navigation counts as loaded when the library cannot read which the query loads: an
include in a form it cannot read, an include off the query's own chain (on a join's inner source, say), and a
chain that reaches its rows through anything but the root's own rows (`Select(o => o.Customer)`, a
`SelectMany`, a `Join`, a `GroupBy`) when it also has an include, which EF Core applies from the root to the
entities it reaches, or when one of its lambdas hands its rows an object: one it builds (a projection behind an
identity `Select`, or an object built inside an anonymous row or a conditional), which loads whatever it
assigns; one an application's method returns from what the lambda gives it; or one it captured, another query
with its own include or projection, or an object in memory. A call that reads nothing of the lambda's and returns
a query or an expression (a specification, a repository's query, `FromSql`, a context's `Set` through an
interface) is evaluated as EF Core evaluates it, and what it returns is read; a context's own query function is a
query root; an anonymous object that only carries what the rows hold (query-syntax range variables, a
composite key) builds nothing; and what only feeds a predicate or a key is a value and hands a row nothing. Such
a chain with none of these is read from the model. A denial beneath a navigation nothing loads never leaves the database
and needs no projection,
so a connected model is read as it was in 3.1.0. A member EF Core does not map counts as loaded: its getter can
hand out a mapped field or a private navigation, so its type is read whole.
- On a row a projection builds: beneath a member its initializer assigns. A constructor with arguments counts
every member as assigned; an initializer after it still says what its own bindings hold.
- On a row in memory: beneath any member.
- So does a member whose value can hold a field denied for Select that no path names (deeper than the walker's
four segments, inside a framework generic such as `Dictionary<string, T>`, or declared by a subtype of its
type), read as for a named member above, so on an entity only what loads counts. Under a policy with a `"*"`
deny, so does a member whose value can hold a path the walk never asks about and the policy does not name.
- A member that can hold an object of any type, one typed `object`, a framework interface such as `IComparable`,
an unbound type parameter, or a collection that is not generic (`IEnumerable`, `ArrayList`, `Array`, an
application's own), asks for nothing on its own: the policy cannot see into it whether or not a projection is
built. An application's own such collection still has its own members read, as any type's are.
- A row can be a subtype of T. On an entity, each member a type the model derives from T declares, and what loads
beneath it, counts as one of T's own would; on a row in memory, each member any loaded subtype declares; on a
row a projection builds, each member the type its initializer constructs declares below T. The projection builds
T, so it leaves them all out, recorded as `Dropped` with a reason starting `left out: a type derived`.
- A subtype is any type loaded outside the framework's own assemblies that derives from the type or implements it:
an open generic one, `Tagged<T> : Creature`, and an application's subclass of a framework class, an `Exception`
or a `Stream`, included. A rule on a path through a subtype's member (`Org.Swift`, where `Swift` is the bank's),
or through one of two members whose names differ only in letter case, counts beneath a member as a rule on the
declared type's own path does.
- The denials beneath a member come from the providers' fragments as well as from walking the type, so a denied
property with no setter, a rule on a path reached through a cycle, and a rule deeper than the walk all count.
- A forced scope beneath a member asks for no projection on its own: it filters the rows that hold the member, as it
always has. When a projection is needed anyway, the member is left out whole (below).
- The projection is what an unguarded call would return, less what the policy withholds:
- every allowed member holding a value, a simple type (primitive, enum, `string`, `decimal`, `DateTime`,
`DateOnly`, `TimeOnly`, `DateTimeOffset`, `TimeSpan`, `Guid`) or a collection of one (`byte[]`, `string[]`,
`List<string>`), that the source carries: every one a projection assigns, every one of a row in memory, and
every one EF Core maps on an entity. A value EF Core does not map is left out: computing it would make EF Core
read the whole entity, the denied columns included, and it holds only its initial value anyway;
- every allowed member holding an object or a list of them that the source carries: a member a projection's
initializer assigns, and an entity's columns (converted or JSON), owned and complex members. It is kept whole
when nothing beneath it is denied, nothing its value can hold is denied, it cannot hold an object of any type
(asked of a projected row, a row in memory, and an entity's column a value converter hands back, directly or
inside a complex property: what EF Core materializes itself never holds one), under a
`"*"` deny every path beneath it the walk skips is one the policy names, no forced scope is beneath it, and no
transform beneath it lands on a property with no setter;
- otherwise it is narrowed to the allowed leaves beneath it, to four segments, as a caller naming it would get,
when the core's narrowing translates: an object the projection's initializer builds, a list a subquery reads
into a type the core can bind (not an array or a set), a navigation that is neither complex nor stored as JSON,
or an entity's owned member not stored as JSON. The narrowing builds the member's declared type, so a
subtype's fields are dropped. A leaf that can hold what the policy cannot name is left out;
- otherwise it is left out whole, recorded as `Dropped` on `Select` with a reason starting `left out whole`: a
scope forced beneath it; a column, complex property or JSON-stored member, which EF Core reads whole; one the
projection builds some other way, by a constructor with arguments, a conditional or an unassigned member; a
denied key the core's projection would add back; a path the core cannot project; a node type the core cannot
construct (an interface, an abstract class, one with no public parameterless constructor); nothing beneath it
left to select;
- Never kept: an entity's navigation, included or not, since projecting it would load it (under Convenience, name
it in `Selects` to get it narrowed; under Strict, name its allowed fields); an object held by a row in memory,
since a kept object is the caller's own and a transform would change it in place; a member with no setter; a
member named with one of the parser's words. Each one the unguarded call would have returned, an included
navigation or an object in memory, is recorded as `Dropped` with a reason starting `left out:` (3.2.0).
- A `Select` handing back an entity, `Select(o => o.Customer)`, builds no row and is read as an entity query, with
every navigation counted as loaded when the query has an include (above).
- A narrowed reference that is null in the source comes back as an empty object, as it does for a caller's own
dotted `Selects`.
- A narrowed member carries every allowed leaf beneath it. An entity reached beneath it has its own navigations
projected and so loaded, which the source may not have included, exactly as `Selects` naming the member does.
- Each denied field whose value can reach the result, at the top or beneath, is recorded as `Dropped` on `Select`.
- It never throws for the denied field. It throws `AllSelectsDenied` only when no field is left. A typed terminal
projects into T, so a T with no public parameterless constructor, an abstract one among them, fails there with
`SelectTypeMustHaveParameterlessConstructor`; the dynamic terminals build their own class.
- With nothing counted, `Selects` stays null and the query is the one an unguarded call runs.
- A path that matches nothing on `T` (3.1.0):
- Convenience, and dry run in either tier: validation throws `LogicException`
`ConditionMustHasValidFieldName` before any policy decision, as it does unguarded.
- Strict, outside dry run: the name is kept and gated as a field denied for every feature, at the step a
denial is raised, after the caps. It throws the code a `[DwDenied]` field throws in the same clause:
`FieldDeniedForWhere`, `FieldDeniedForSelect`, `FieldDeniedForOrder`, `FieldDeniedForGroup` or
`FieldDeniedForAggregate`, and `FieldDeniedForSegment` in any clause of a Segment.
- A name padded with dots or blank segments is normalized as a real path is, empty segments dropped (3.1.0):
`NoSuchColumn....`, `....X` and `. . . . X` are gated as `NoSuchColumn` and `X`, and refused as a padded real
field is. Kept whole, it would fail `MaxNavigationDepth` where a padded real field passes.
- It covers condition fields, `Selects`, `Orders`, `GroupBy.Fields` and `AggregateBy.Field`. `Having` and
summary `Orders` name aliases and group keys, which validation still checks. A null or blank name is
refused the same way in both tiers.
- Under `Strict`, field and cap refusals name no field (3.1.0). Every `FieldDeniedFor*` refusal carries
`FieldPath = "*"`, `RuleId = null` and `SourceOrigin = null`, for a denied field, an alias and an unknown name
alike, so their messages are identical. Every `CapExceeded` refusal carries `FieldPath = "*"` as well, and
keeps its `SourceOrigin`. `MissingContextValue` carries `FieldPath = "*"` and `SourceOrigin = null`, naming
neither the scope's column nor the context key it reads. `OperatorNotAllowed`, `RequiredFilterMissing` and
`AmbiguousFieldName` still name their field.
- Inside a `Segment` every field refusal is `FieldDeniedForSegment` with `Feature = Segment`, whichever clause
refused it: a condition in any set, an order, a select, or taking part at all (3.1.0). Answered by clause, a field
denied for every clause but not for `Segment` would say `FieldDeniedForOrder` where a name that matches nothing
says `FieldDeniedForSegment`. The Convenience tier keeps per-clause codes, and filters and summaries keep them in
both tiers.
- The cost budget is checked after every field gate (3.1.0). A weighted field the caller may not use is refused
as denied before its weight counts, exactly as a name that matches nothing is; an allowed weighted field still
gets `QueryCostExceeded`.
- The trace keeps the real path and reason; an unknown name is recorded as `Denied` with reason
`names nothing on <TypeName>`. A refusal event under `AuditRefusals` names the field too (section 22).
- Convenience names the field as the caller wrote it, with `RuleId` and `SourceOrigin`.
- "left out" (3.1.0): a guarded query that sends no orders takes the type's `DefaultOrder` less every field
this caller may not order by, in both tiers, and in a `Segment` less every field this caller may not use in a
segment. Each field left out is recorded as `Dropped` on `Order`, with a reason starting
`left out of the default order`, and never refused. Dry run keeps it and still records it (section 13).
- These throw regardless of tier and dry run: `AmbiguousFieldName`, `GroupTooSmall`, `AmbiguousGroupKey`,
`CapExceeded` from a full audit buffer, `MissingHashSalt`, `MissingTokenVault`,
`TransformRequiresMaterialization`, `StoreUnavailable` and `PolicyContextNotPrepared`. `PolicyRequired`
is raised on unguarded calls only.
- Dry run is on when `DwPolicyOptions.DryRun` or `DwPolicyContext.DryRun` is true. It records every other
throw and drop in the trace, applies none of them, and leaves the clause as written. In dry run:
- no forced predicate is injected and no projection is synthesized;
- the group-size column is added without its predicate;
- a default order keeps the fields the caller may not order by;
- a path that matches nothing fails validation with `ConditionMustHasValidFieldName`, in both tiers;
- output transforms still run.
- a dry run therefore returns what the policy would have withheld: denied fields, and rows a forced
predicate would have excluded. Only the transforms still apply.
---
## 18. Transforms
Transforms run in memory after rows materialize, on the returned instances. Every guarded query runs
`AsNoTracking`. WHERE, ORDER BY, GROUP BY and aggregates run in SQL on the real values.
```
Order TransformKind Attribute Output Valid member type
alone Default [DwDefault] constant or type default, converted to member any
1 Mutate [DwMutate] whatever Transform returns any it fits
2 Generalize [DwGeneralize] Round/Truncate/DatePart keep the value's type; numeric; DateTime,
Bucket returns string DateOnly, DateTimeOffset;
Bucket: string
3 Format [DwFormat] string string
4 Mask [DwMask] string; Null strategy returns null string; Null: reference or T?
5 Truncate [DwTruncate] string string
```
- `[DwDefault]` short-circuits the chain: no other stage runs. Declaring it with any other transform
attribute is a `ValidateModel` error.
- The chain's result must be assignable to the member's type, and null only fits a reference type or
`Nullable<T>`. Otherwise the query throws `InvalidOperationException`, not `PolicyException`.
- So a mask, format, truncate or `Bucket` on a `decimal` or `DateTime` member fails.
- On such a member, use `[DwGeneralize]` (not `Bucket`) or `[DwDefault]`.
- Mask and Truncate read the value as text: a `string` as-is, an `IFormattable` via
`ToString(null, CultureInfo.InvariantCulture)`, anything else via `ToString()`. Null stays null, except
under `Fixed`.
- Only paths the result carries are transformed: every path when `Selects` is null, otherwise the
selected paths and every path beneath a selected navigation.
- Collections and navigations are followed, and null navigations are skipped.
- An object shared by several rows is transformed once.
- A transformed member with no setter, or a path missing on the runtime type, throws
`InvalidOperationException`.
- Segment results are transformed like Filter results.
- In a summary, each transformed grouping-key column and each aliased aggregate of a transformed field
gets that field's chain.
- The trace gets one `PolicyDecision` per transformed path. Its feature is `Select`, or `Aggregate` for
summary columns, and its reason names the stages, e.g. `"Generalize then Mask"`.
- Aggregating a transformed field throws `FieldDeniedForAggregate` unless every declared stage has
`AllowAggregate = true`. That includes a short-circuiting `[DwDefault]`, and the check overrides every
Allow, sealed ones included. The field's own floor is the largest `MinGroupSize` among its stages.
- On the guarded handle, `SelectDynamic`, `FilterDynamic`, `Group` and `Summary` throw
`TransformRequiresMaterialization` if the type has any transform for this caller.
- `AsUnguardedQueryable()` output is never transformed.
- Transforms also run in dry run.
### DwMask
```
Strategy Output for non-null text v null input
Full MaskChar repeated v.Length times (8 times when PreserveLength = false) null
Partial v[..KeepStart] + one MaskChar per hidden char + v[^KeepEnd..] null
if KeepStart + KeepEnd >= v.Length: Full
Email local[0] + n MaskChar + "@" + domain[0] + m MaskChar + domain from its last "." null
PreserveLength: n = max(local.Length - 1, 1), m = max(lastDotIndex - 1, 1)
PreserveLength = false: n = m = 3
Full when: no "@", "@" first or last, no "." after the domain's first char, "." last
Phone every digit masked except the last K (K = KeepEnd, or 4 when KeepEnd = 0) null
non-digits kept; K or fewer digits: Full
Regex Regex.Replace(v, Pattern, Replacement ?? "", RegexOptions.None, 1-second timeout) null
Fixed Text Text
Hash lowercase hex of HMAC-SHA256(key = UTF-8 HashSalt, data = UTF-8 v), 64 chars null
Null null null
Tokenize TokenVault.GetOrCreate(TokenScope ?? field path, v); built-in vaults: 32 lowercase hex null
```
```
Partial KeepEnd = 4 "999999991234" -> "********1234"
Partial KeepEnd = 4 "1234" -> "****"
Partial KeepStart = 2, KeepEnd = 2 "abc" -> "***"
Email "ada@example.com" -> "a**@e******.com"
Email PreserveLength = false "ada@example.com" -> "a***@e***.com"
Email "ada@example" -> "***********"
Phone "+964 770 12 1234" -> "+*** *** ** 1234"
Phone "123" -> "***"
Regex Pattern = @"\d", Replacement = "#" "A1-23" -> "A#-##"
Fixed Text = "[REDACTED]" null -> "[REDACTED]"
Full "Secret123" -> "*********"
Full PreserveLength = false "Secret123" -> "********"
Email "john.doe@example.com" -> "j*******@e******.com"
Phone "07701234567" -> "*******4567"
Hash any value -> 64 lowercase hex characters
Tokenize any value -> 32 lowercase hex characters
```
- `PreserveLength` affects `Full`, `Email`, and the Full fallback of every strategy. Partial's hidden
middle and Phone's digits are always masked one character for one.
- Refused when the stage is built (a `ValidateModel` error, or `ArgumentException` at query time):
- a negative `KeepStart` or `KeepEnd`;
- `Regex` with a null or empty `Pattern`;
- `Fixed` with `Text = null` (`""` is allowed);
- a blank `TokenScope`.
Invalid regex syntax surfaces only when a query runs.
- `Hash` without `DwPolicyOptions.HashSalt` throws `MissingHashSalt`. The salt is checked before the value,
so even a null value throws.
- The `HashSalt` setter refuses 1–15 characters. Blank means not configured.
- The same value under the same salt always gives the same digest.
- `Tokenize` without `DwPolicyOptions.TokenVault` throws `MissingTokenVault`, also checked before the
value. A null value is not given a token.
- The default token scope is the canonical path relative to the queried entity, e.g. `"NationalId"` or
`"Contact.NationalId"`. The entity type is not part of it. The same scope and value always give the same
token.
- `DwToken.New()` returns 16 cryptographic random bytes as 32 lowercase hex characters.
- `DwToken.KeyFor(string scope, string value)` returns `scope + ":" + lowercase hex SHA-256(UTF-8 value)`.
`InMemoryTokenVault`, `RedisTokenVault` and `EfTokenVault` all store mappings under this key. There is no
reverse lookup.
- `InMemoryTokenVault` lasts for the process and is unbounded. It exposes `Count` and `Clear()`, and a
restart issues new tokens.
- `Hash` and `Tokenize` both preserve equality, so the column still groups and joins, given the same salt
or the same scope.
### DwGeneralize
```
Mode Requires Output Valid on
Round Step > 0 Math.Round(v / Step, MidpointRounding.AwayFromZero) * Step, same type numeric
Bucket Step > 0 "{lo}-{lo+Step-1}" with lo = Math.Floor(v / Step) * Step, invariant string holding a number
DatePart Part start of the Year / Quarter / Month / Day DateTime, DateOnly,
DateTimeOffset
Truncate Decimals >= 0 Math.Truncate(v * 10^Decimals) / 10^Decimals, same type, no rounding numeric
```
```
Round Step = 10000 118500 -> 120000
Round Step = 10 23 -> 20 -14 -> -10
Bucket Step = 10 27 -> "20-29" 30 -> "30-39" -1 -> "-10--1"
Truncate Decimals = 2 33.199999 -> 33.19
DatePart, 1987-06-15 Year 1987-01-01 Quarter 1987-04-01 Month 1987-06-01 Day 1987-06-15
```
- A null value stays null. Arithmetic runs in `decimal` and converts back to the value's own type, so an
`int` stays an `int`.
- `DatePart` keeps a `DateTime`'s `Kind`, returns a `DateOnly` for a `DateOnly`, and keeps a
`DateTimeOffset`'s `Offset`.
- Round, Bucket and Truncate use `Convert.ToDecimal(value, InvariantCulture)`. DatePart uses
`Convert.ToDateTime` for anything that is not a `DateOnly` or `DateTimeOffset`.
- A value of the wrong kind throws when the query runs. `ValidateModel` checks only Bucket's
string-member rule.
### DwFormat, DwTruncate, DwDefault
- `[DwFormat]` renders an `IFormattable` value with `value.ToString(Format, CultureInfo.InvariantCulture)`.
Any other value uses `ToString()` and ignores `Format`, so a string passes through unchanged.
- On a string member it therefore changes only a value that `[DwMutate]` or `[DwGeneralize(DatePart)]`
turned into a non-string. `Round` and `Truncate` convert back to string.
- A blank format is refused.
- `[DwTruncate]` leaves a null value, or a value of at most `Length` characters, unchanged. A longer value
becomes `v[..Length]` followed by `Ellipsis` when `Ellipsis` is not null.
- `Ellipsis` is not counted in `Length`: the output can be `Length + Ellipsis.Length` characters.
- `Length = 0` is allowed; a negative length is refused. Truncation runs after the mask.
- `[DwDefault]` and `[DwDefault(null)]` give the member type's default: `0`, `false` or a default struct
for a non-nullable value type, otherwise null.
- `[DwDefault("v")]` converts the text to the member type, after unwrapping `Nullable<T>`:
- `string` → `"v"` unchanged;
- `""` on any other type → the type's default;
- enum → `Enum.Parse(ignoreCase: true)`;
- `Guid` → `Guid.Parse`;
- `DateOnly`, `TimeOnly`, `DateTimeOffset`, `TimeSpan` → `.Parse` with `InvariantCulture`;
- anything else → `Convert.ChangeType` with `InvariantCulture`.
A constant that cannot convert is a `ValidateModel` error; at query time the parse exception is thrown.
### DwMutate
```csharp
public interface IValueTransformer
{
object? Transform(object? value, DwTransformContext context);
}
public readonly struct DwTransformContext : IEquatable<DwTransformContext>
{
public DwTransformContext(object entity, string fieldPath, DwPolicyContext policy); // null -> ArgumentNullException
public object Entity { get; } // object holding the member: nested owner for "Contact.Email", generated row in a summary
public string FieldPath { get; } // canonical path, never the alias
public DwPolicyContext Policy { get; } // the caller: Subjects, Identities(DwSubjectKind), Purpose, TryGetValue
}
```
- One instance per transformer `Type` is cached for the life of the process. It comes from
`DwPolicyOptions.Services?.GetService(type)`, falling back to `Activator.CreateInstance(type)`, which
needs a parameterless constructor. The implementation must be stateless and thread-safe.
- `[DwMutate]` runs first and sees the real value. Exceptions it throws are not caught, so the query fails.
- A type that does not implement `IValueTransformer` throws `InvalidOperationException` at query time, and
is a `ValidateModel` error.
- The transformer's result must fit the member. `ValidateModel` skips the output-type check for any chain
containing `[DwMutate]`.
- It is available from source only: a stored rule cannot carry a Mutate stage.
### Stage classes
For runtime rules and custom providers. Each constructor throws `ArgumentException` for the same
misconfigurations as the matching attribute.
```
TransformStage (abstract) Kind:TransformKind AllowAggregate:bool MinGroupSize:int
MutateStage(Type transformer, bool allowAggregate = false, int minGroupSize = 0)
GeneralizeStage(GeneralizeMode mode, int step = 0, DatePart part = DatePart.Year, int decimals = 0,
bool allowAggregate = false, int minGroupSize = 0)
FormatStage(string format, bool allowAggregate = false, int minGroupSize = 0)
MaskStage(MaskStrategy strategy, int keepStart = 0, int keepEnd = 0, char maskChar = '*',
bool preserveLength = true, string? pattern = null, string? replacement = null, string? text = null,
bool allowAggregate = false, int minGroupSize = 0, string? tokenScope = null)
Replacement:string (a null replacement becomes "")
TruncateStage(int length, string? ellipsis = null, bool allowAggregate = false, int minGroupSize = 0)
DefaultStage(string? value, bool hasValue, bool allowAggregate = false, int minGroupSize = 0)
ValueTransform(MutateStage? mutate = null, GeneralizeStage? generalize = null, FormatStage? format = null,
MaskStage? mask = null, TruncateStage? truncate = null, DefaultStage? @default = null)
Mutate Generalize Format Mask Truncate Default AllowsAggregate MinGroupSize IsEmpty
HasConflictingDefault Stages (in run order; a Default runs alone) Action
```
---
## 19. Group floor and transformed summaries
```
effective floor = max(Caps.MinGroupSize, MinGroupSize of the chain of each AggregateBy.Field)
Caps.MinGroupSize unset -> DwCaps.DefaultMinGroupSize = 5 (IsMinGroupSizeSet = false); 1 = no global floor
```
- The floor applies to every guarded `Summary` that has a `GroupBy`, once it is above 1, whether or not
any field is transformed. Only aggregated fields raise it; grouping keys do not.
- It is added after gating and is not charged against cost:
- `AggregateBy { Aggregator = Count, Alias = "__dwGroupSize" }` is appended.
- `Having` becomes `And [ __dwGroupSize >= floor ]`, with the caller's `Having` as a subgroup.
- The database removes the small groups, so `Data`, `TotalCount` and `PageCount` cover only the groups
that remain.
- A summary whose every group is too small returns an empty `Data`, never an error.
- The trace gets one decision: `__dwGroupSize`, `Aggregate`, `Dropped`,
`"groups below the group floor of {floor} are excluded by the query"`.
- `ToList`/`ToListAsync(Summary)` return each row as an `ExpandoObject` without `__dwGroupSize` whenever
the floor applied.
- The composable `Group(GroupBy)` and `Summary(...)` apply the floor as well, and project the column back out,
so no caller receives the library's own count. `Group` reaches the floor by running the summary pipeline.
- `GroupTooSmall` means the caller already used `"__dwGroupSize"` (case-insensitive) as an aggregate
alias, a `Having` field or an `Orders` field. It throws in both tiers and in dry run. A small group never
raises it.
- Dry run adds the column without the predicate. Each group below the floor gets its own decision
(`"a group of {n}, below the group floor of {floor}"`) and stays in `Data`.
- Transformed keys are handled in this order:
- Groups form in SQL on real values, and small groups are removed.
- Each transformed grouping-key column (named as the key path without dots) and each aliased aggregate
of a transformed field gets the field's chain. `MAX(Age) = 41` with `Round, Step = 10` returns `40`.
- If two rows now share the same values across the transformed keys, the query throws
`AmbiguousGroupKey`, in both tiers and in dry run. `FieldPath` is the transformed key paths joined by
`", "` and `Feature` is `Group`.
- Only the transformed keys are compared. Grouping by `[Department, Salary]` with `Salary` rounded throws
as soon as two departments share a rounded salary.
---
## 20. Model validation
```
DwPolicy.ValidateModel(params Type[] types) -> PolicyModelReport
DwPolicy.ValidateModel(DwPolicyOptions? options, params Type[] types) -> PolicyModelReport
throws InvalidOperationException listing every error, if there is any
PolicyModelValidator.Inspect(IEnumerable<Type> types) -> PolicyModelReport (never throws)
PolicyModelValidator.Inspect(IEnumerable<Type> types, DwPolicyOptions? options)
PolicyModelReport Errors:IReadOnlyList<string> Warnings:IReadOnlyList<string> IsValid (no errors)
```
```csharp
DwPolicyOptions options = new() { HashSalt = secret, TokenVault = vault };
options.Entities.Expose<Employee>("Employee");
PolicyModelReport report = DwPolicy.ValidateModel(options, options.Entities.ToArray()); // throws when invalid
foreach (string warning in report.Warnings) logger.LogWarning("{Warning}", warning);
DwPolicy.Configure(options, providers);
```
- Validation never runs automatically.
- It reads only the public instance properties of the listed types, so pass DTO and navigated types too.
- It does not check runtime rules.
- Each message starts with `Type.Member: `, except the `DefaultOrder` messages, which start with `Type: `.
Errors:
- two members of one type with the same `[DwAlias]`, compared case-insensitively
- `[DwDescribe]` that sets nothing
- `[DwAllowedValues]` with no values
- a negative `[DwCost]`
- `[DwAudit]` with `None` or an undefined bit
- a `[DwForceWhere]` that resolving the type's policy would refuse, with that refusal's message (3.1.0; before,
it surfaced only on the first guarded query):
- neither or both of `Value` and `ContextValue`, or either one with `IsNull` / `IsNotNull`
- a member type with no `DataType` (section 14)
- `AllowNull = true` with `IsNull` / `IsNotNull`, or on a member that can never be null
- `[DwEntity(DefaultOrder = ...)]` (3.1.0):
- an entry that is not a field optionally followed by `asc` or `desc`:
`"{Type}: DefaultOrder entry '{entry}' is not a field optionally followed by asc or desc, so guarded queries skip it."`
- a field whose name begins with one of the parser's own words (section 5), judged before the type is asked
whether it has the member, because the type may well have it:
`"{Type}: DefaultOrder names '{field}', which starts with a name the expression parser keeps for itself, so no query can use it. Rename the member."`
- a field no query can order by, a path ending on a collection of entities:
`"{Type}: DefaultOrder names '{field}', which no query can order by, so guarded queries skip it."`
- a field the type's own attributes deny for ordering, unless every one of those denials is `Overridable` (then a
warning, below). The attributes include those of the member's other declarations, an interface member it
implements, a subtype's override and a public member a subtype hides with `new` (3.2.0):
`"{Type}: DefaultOrder names '{field}', which its attributes deny for ordering, so every guarded query leaves it out."`
- a stage that cannot be built:
- `Round` or `Bucket` with `Step <= 0`, or `Truncate` with `Decimals < 0`
- a blank `[DwFormat]`
- a negative `KeepStart` or `KeepEnd`
- `Regex` without `Pattern`, or `Fixed` without `Text`
- a blank `TokenScope`
- a negative `[DwTruncate]` length
- `[DwDefault]` together with another transform attribute
- `[DwMutate]` naming a type that does not implement `IValueTransformer`
- when `options` is passed: `Hash` with an empty `HashSalt`, or `Tokenize` with a null `TokenVault`
- output type (skipped for chains with `[DwMutate]`):
- a `[DwDefault("v")]` that cannot convert
- `MaskStrategy.Null` on a non-nullable value type
- `Format`, `Truncate`, `Bucket`, or any mask other than `Null`, on a member that is not `string`
(after unwrapping `Nullable<T>`)
Warnings:
- the member has a transform other than `[DwDefault]`, and no DwDeny-family attribute on it covers `Order`.
The message ends "Add [DwNoOrder] unless that is intended."
- `DefaultOrder` names a field the type does not have (3.1.0):
`"{Type}: DefaultOrder names '{field}', which {Type} does not have, so guarded queries skip it."`
- `DefaultOrder` names a field denied for ordering only by overridable attributes on the member its path ends on,
such as `[DwNoOrder(Overridable = true)]`, which a rule can lift for some callers (3.1.0):
`"{Type}: DefaultOrder names '{field}', which its attributes deny for ordering unless a rule allows it, so guarded queries leave it out until one does."`
- `DefaultOrder` names a field the attributes allow ordering but deny for segments (3.1.0):
`"{Type}: DefaultOrder names '{field}', which its attributes deny for segments, so guarded segments leave it out."`
Not checked; these throw at query time:
- a blank, dotted or `"*"` alias
- a blank describe value or a blank listed value
- an invalid regex
- `Round`, `Truncate` or `DatePart` on a value of the wrong kind
Attributes on fields are neither checked nor applied.
---
## 21. Trace, explain and custom providers
### PolicyTrace
```
PolicyTrace sealed class, DynamicWhere.ex.Policies.DTOs
PolicyTrace(DwTier tier, bool dryRun)
Tier : DwTier
DryRun : bool true when this query ran in dry run
Decisions : IReadOnlyList<PolicyDecision> in the order taken; read-only view
Add(PolicyDecision decision) null → ArgumentNullException
PolicyDecision sealed class
PolicyDecision(string fieldPath, PolicyFeature feature, PolicyAction action, string? reason) blank path → ArgumentException
FieldPath : string canonical path, never the alias typed; "*" = whole request; "__dwGroupSize" = group floor;
under Strict, a name that matches nothing as the caller sent it, trimmed and with
empty segments dropped (3.1.0)
Feature : PolicyFeature
Action : PolicyAction
Reason : string?
PolicyAction Allowed=0 Denied=1 Dropped=2 Masked=3 Injected=4 Mutated=5 Defaulted=6 Generalized=7
PolicyFeature None=0 Where=1 Select=2 Order=4 Group=8 Aggregate=16 Segment=32 All=63 [Flags]
```
Where to read a trace:
- `FilterResult<T>.Policy` or `SummaryResult.Policy` (`SegmentResult<T>` inherits it). It is null on unguarded calls, and on guarded ones when `DwPolicyOptions.IncludeTraceInResult` withholds it: by default under `Strict` (3.1.0).
- `PolicyQueryable<T>.LastTrace`.
- `PolicySimulation<TClause>.Trace`.
A refused call throws, and `PolicyException` carries no trace. To see a refusal's decisions, run it in dry run or through `PolicySimulator`. A Strict refusal of a name that matches nothing shows only through `PolicySimulator`, because dry run fails that name in validation instead.
```
What is recorded
Dropped a field removed from a Convenience request (Select, Order); one per field
Dropped a DefaultOrder field left out for this caller (Order), both tiers, and in a Segment a field
denied for Segment too; Reason starts "left out of the default order" (3.1.0)
Dropped a denied field a synthesized projection leaves out (Select), at the top or beneath a member;
Reason names the policy's sources, such as "DwDeniedAttribute (sealed)" (3.2.0)
Dropped a member a synthesized projection leaves out whole (Select); Reason starts "left out whole" (3.2.0)
Dropped a member a type derived from T declares, which a synthesized projection building T leaves out
(Select); Reason starts "left out: a type derived" (3.2.0)
Dropped a member a synthesized projection cannot keep that the unguarded call would have returned, an
included navigation or an object of a row in memory (Select); Reason starts "left out:" (3.2.0)
Dropped a member a Convenience caller named that can hold a denied field no path names (Select); Reason
"it holds a field denied for Select where no path can name it" (3.2.0)
Denied a refusal: Strict denial, cap, cost, query string ("*"), required filter, segment inference,
MaxAuditEvents; the throw follows unless dry run. Under Strict a name that matches nothing
is Denied under that name, Reason "names nothing on <TypeName>" (3.1.0)
Injected a forced predicate added (Where); Reason "forced predicate (<Operator>)", or
"forced predicate (<Operator>, or null)" when the term admits null (3.1.0)
Masked | Mutated | Defaulted | Generalized
once per transformed path per query (Select); Reason lists the stages ("Mask then Truncate");
summary key and aggregate columns use Feature Aggregate
Allowed an aliased column renamed on output (dynamic and Summary rows); Reason "emitted as '<alias>'"
Dropped "__dwGroupSize", Aggregate: the floor was applied; in dry run, one record per group below it
```
- A chain records `Defaulted` if it has a `[DwDefault]` stage, else `Mutated`, else `Masked`, else `Generalized`. A Format/Truncate-only chain records `Masked`.
- A Convenience drop leaves no marker in `Data`. The trace is the only way to tell a dropped field from a null value.
### PolicyResolver and explanations
```
PolicyResolver sealed class, DynamicWhere.ex.Policies.Resolution
PolicyResolver(IEnumerable<IDwPolicyProvider> providers) null → ArgumentNullException; null element → ArgumentException
Resolve(Type entityType, string fieldPath, DwPolicyContext context) -> FieldPolicy
Explain(Type entityType, string fieldPath, DwPolicyContext context) -> PolicyExplanation
ResolveType(Type entityType, DwPolicyContext context) -> TypePolicy
```
```csharp
PolicyExplanation why = DwPolicy.Resolver.Explain(typeof(Employee), "Salary", caller);
```
- `PolicyResolver.Explain` is the only explain API in the core package.
- `new PolicyResolver(...)` does not add `AttributePolicyProvider`; only `DwPolicy.Configure` does. Include it yourself, or attributes are ignored.
- `fieldPath` must be the canonical property path. It is trimmed, empty segments are dropped, and matching is case-insensitive.
- An alias is not translated, and the path is not checked for existence.
- Either mistake explains as Allow for every feature, because only wildcard fragments match.
- Errors:
- null type or context → `ArgumentNullException`;
- blank path → `ArgumentException`;
- a provider returning null or a null fragment → `InvalidOperationException` naming the provider.
- Resolution uses the context: with a store configured, an unprepared context throws `PolicyContextNotPrepared`.
- Explaining records no audit events.
```
PolicyExplanation sealed class, DynamicWhere.ex.Policies.DTOs
EntityType : string Type.FullName
FieldPath : string canonical
Policy : FieldPolicy the decision Resolve returns
Features : IReadOnlyList<FeatureExplanation> Where, Select, Order, Group, Aggregate, Segment, in that order
FeatureExplanation sealed class
Feature : PolicyFeature
Effect : PolicyEffect Allow when nothing spoke
DecidedBy : PolicySource? null when nothing spoke
Level : PolicyLevel? null when nothing spoke
TiedWith : IReadOnlyList<PolicySource> equal to the winner on level, specificity, priority and effect
Overrode : IReadOnlyList<PolicySource> outranked and discarded, not merged
IsAttributionAmbiguous : bool TiedWith.Count > 0; DecidedBy is one of several equals
PolicySource sealed class
Origin : string attribute type name, or "Rule <ruleId>"
RuleId : string? null for an attribute
Subject : string? null for an attribute
IsSealed : bool
static FromAttribute(string attributeName, bool isSealed) -> PolicySource blank name → ArgumentException
static FromRule(string ruleId, string subject) -> PolicySource blank argument → ArgumentException
ToString() "<Origin>", "<Origin> (sealed)", or "<Origin> [<Subject>]" for a rule
PolicyEffect Allow=0 Mask=1 Deny=2
PolicyLevel SealedAttribute=1 DynamicUser=2 DynamicRole=3 DynamicTenant=4 DynamicGlobal=5 OverridableAttribute=6
```
- `Effect` includes the resolver's own override. A transformed field without `AllowAggregate` reports `Aggregate` as `Deny`, even when `DecidedBy` is null or an Allow fragment.
- Fragments that decide no feature (an alias or facts only) appear in neither `TiedWith` nor `Overrode`.
```
FieldPolicy sealed class
FieldPath : string Sources : IReadOnlyList<PolicySource> IsSealed : bool
EffectFor(PolicyFeature feature) -> PolicyEffect Allow when no fragment spoke
Allows(PolicyFeature feature) -> bool anything but Deny
IsMasked(PolicyFeature feature) -> bool effect is Mask
AllowedOperators : IReadOnlyList<Operator>? null = unrestricted, empty = none allowed
AllowsOperator(Operator op) -> bool
Alias : string? ForcedPredicates : IReadOnlyList<ForcedPredicate>
RequiredOperators : IReadOnlyList<Operator>? IsRequiredInWhere : bool SatisfiesRequirement(Operator op) -> bool
Transform : ValueTransform? IsTransformed : bool
Facts : FieldFacts? Label, Description, Group : string? Order : int? AllowedValues : IReadOnlyList<string>?
CostWeight : int? AuditedFeatures : PolicyFeature? IsAudited() -> bool IsAudited(PolicyFeature feature) -> bool
TypePolicy sealed class
Aliases : IReadOnlyDictionary<string, IReadOnlyList<string>> public name → canonical paths
Forced : IReadOnlyList<ForcedPredicate>
Required : IReadOnlyDictionary<string, IReadOnlyList<Operator>>
Transforms : IReadOnlyDictionary<string, ValueTransform>
IsEmpty : bool
```
### Custom policy providers
```
IDwPolicyProvider interface, DynamicWhere.ex.Policies.Resolution
IReadOnlyList<PolicyFragment> GetFragments(Type entityType, DwPolicyContext context);
AttributePolicyProvider : IDwPolicyProvider sealed class; public parameterless constructor; const int MaxDepth = 4
```
Pass providers in:
```
DwPolicy.Configure(options, providerA, providerB)
builder.Services.AddDwPolicies(section, configure, providerA, providerB)
new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider(), providerA })
// for the explicit ApplyPolicy, PolicySimulator and PolicySchemaBuilder
```
- `GetFragments` runs synchronously on the query path, per field per query. Do no I/O; serve from memory.
- Return every fragment for the type; the resolver does the path matching.
- Return an empty list, never null and never a null element (either → `InvalidOperationException`).
- The resolver trusts the `Level` a fragment claims, `SealedAttribute` included.
- `DwPolicy.PrepareAsync` prepares only `StorePolicyProvider` instances. A custom provider gets no per-request async hook.
```
PolicyFragment sealed class, DynamicWhere.ex.Policies.DTOs
PolicyFragment(string fieldPath, PolicyFeature features, PolicyEffect effect, PolicyLevel level,
PolicySource source, int priority = 0, object? payload = null,
IReadOnlyList<Operator>? allowedOperators = null, string? alias = null,
ForcedPredicate? forced = null, IReadOnlyList<Operator>? requiredOperators = null,
TransformStage? transform = null, FieldFacts? facts = null)
const string Wildcard = "*"
FieldPath Features Effect Level Source Priority Payload AllowedOperators Alias Forced
RequiredOperators Transform Facts get-only; FieldPath normalized, Alias trimmed
IsWildcard : bool
Matches(string fieldPath) -> bool wildcard, or equal ignoring case
Covers(PolicyFeature feature) -> bool every bit of feature is in Features
static NormalizePath(string fieldPath) -> string
ForcedPredicate sealed class
static FromConstant(string fieldPath, Operator op, DataType dataType, string value) -> ForcedPredicate
static FromConstant(string fieldPath, Operator op, DataType dataType, string value,
bool allowNull) -> ForcedPredicate 3.1.0
static FromContext(string fieldPath, Operator op, DataType dataType, string contextValue) -> ForcedPredicate
static FromContext(string fieldPath, Operator op, DataType dataType, string contextValue,
bool allowNull) -> ForcedPredicate 3.1.0
static FromNullCheck(string fieldPath, Operator op, DataType dataType) -> ForcedPredicate op: IsNull | IsNotNull
FieldPath Operator DataType Value ContextValue ReadsContext IsNullCheck AllowNull (3.1.0)
FieldFacts sealed class
FieldFacts(string? label = null, string? description = null, string? group = null, int? order = null,
IReadOnlyList<string>? allowedValues = null, int? costWeight = null, PolicyFeature? auditedFeatures = null)
static ForCost(int weight) static ForAudit(PolicyFeature features) static ForLabel(string label)
Label Description Group Order AllowedValues CostWeight AuditedFeatures Describes
```
- `PolicyFragment` → `ArgumentException` for:
- a blank path;
- an alias that is blank, dotted or `"*"`;
- on the wildcard: an alias, descriptive facts (label, description, group, order, allowed values), `requiredOperators` or a `transform`;
- a field fragment whose `forced` names another field.
- A null `source` → `ArgumentNullException`.
- `FieldFacts` → `ArgumentException` when nothing is supplied, for blank text, for empty or blank `allowedValues`,
or for `auditedFeatures` None or an unknown bit; `ArgumentOutOfRangeException` for `costWeight < 0` (0 is allowed).
- `ForcedPredicate` factories → `ArgumentException` for a blank `fieldPath`, `value` or `contextValue`;
`FromNullCheck` takes only `IsNull` / `IsNotNull`.
- `FromConstant` and `FromContext` with `allowNull: true` and `IsNull` or `IsNotNull` → `ArgumentException`
("AllowNull widens a comparison, and a null check compares against nothing.") (3.1.0). A null check ignores
a constant, and a widened `IsNotNull` would render `(field IS NOT NULL OR field IS NULL)`: a scope that scopes nothing.
Without `allowNull`, `FromConstant` accepts a null check and ignores its value, as before.
- `FromContext` with `IsNull` or `IsNotNull` → `ArgumentException`, whatever `allowNull` says (3.1.0). Without
`allowNull` the message is "'IsNull' compares against nothing, so it reads no context value; build it with
FromNullCheck."; with it, the refusal above answers first. A null check has nowhere to put a context value, so
the key is refused where the predicate is built.
- Fixed in 3.1.0. The factory used to accept it, and the key was still required: a caller without it was refused
with `MissingContextValue`, and a caller with it had the value added to a null check that validation refuses
(`ConditionWithOperator[IsNull-IsNotNull]MustHasNoValues`). Every guarded query on the type failed.
- `AllowNull` (3.1.0): true injects `(field op value OR field IS NULL)` in a group of its own (section 14). The
four-argument `FromConstant` and `FromContext` mean `allowNull: false`, and `FromNullCheck` never sets it.
The factories know no member type, so nothing refuses `allowNull` on a member that can never be null. On a
non-nullable value-type member of the queried type such a predicate injects the comparison alone, which is
the same predicate, and its trace reason has no ", or null"; a path through a navigation is always widened,
because the navigation can be absent.
- Transform stages derive from `TransformStage` and each has a public constructor: `MutateStage`, `GeneralizeStage`,
`FormatStage`, `MaskStage`, `TruncateStage`, `DefaultStage` (signatures in section 18).
---
## 22. Audit
```
IDwAuditSink interface, DynamicWhere.ex.Policies.Audit
ValueTask WriteAsync(DwAuditEvent auditEvent, CancellationToken ct = default);
DwAuditEvent sealed class
DwAuditEvent(DateTimeOffset occurredAt, string entityType, string fieldPath, PolicyFeature feature,
PolicyEffect effect, IReadOnlyList<DwSubject> subjects, string? purpose, DwTier tier, bool dryRun)
DwAuditEvent(DateTimeOffset occurredAt, string entityType, string fieldPath, PolicyFeature feature,
PolicyEffect effect, IReadOnlyList<DwSubject> subjects, string? purpose, DwTier tier, bool dryRun,
PolicyErrorCode? errorCode) 3.1.0
both: blank entityType or fieldPath → ArgumentException; null subjects → ArgumentNullException
OccurredAt : DateTimeOffset UTC, when recorded
EntityType : string Type.FullName
FieldPath : string canonical path, never the alias typed; for a refusal, see below
Feature : PolicyFeature the single use: Where, Select, Order, Group, Aggregate or Segment;
for a refusal, the refusal's Feature
Effect : PolicyEffect the policy's effect for that feature, whether or not the query then ran;
Deny for a refusal
Subjects : IReadOnlyList<DwSubject> the context's subjects
Purpose : string? DwPolicyContext.Purpose
Tier : DwTier
DryRun : bool false for a refusal
ErrorCode : PolicyErrorCode? 3.1.0. The refusal an event records; null for a use of an audited field
ToString() "<OccurredAt:O> <EntityType>.<FieldPath> <Feature> <Effect> <ErrorCode> [<subjects>]"
the code only when not null, the subjects only when there are any
```
- An event is recorded each time a guarded query resolves a field for a feature in the field's audited set.
- `[DwAudit]` defaults to `PolicyFeature.All`; rules can add features.
- One event per reference: a field in a condition and in an order gives two events.
- A field the type's `DefaultOrder` adds is a use too (3.1.0): audited for `Order`, it is recorded as an `Order`
use, `Effect` `Allow`, each time a guarded query orders by it, as a caller's own order is. A dry run keeps a
default field that enforcement would leave out, so it records that one too, with its `Order` effect (`Deny` for a
field the caller may not order by) and `DryRun` true.
- Events are recorded whether the use was allowed, masked, dropped or denied, and in dry run too.
- Nothing is recorded for:
- a `DefaultOrder` field left out for this caller, outside a dry run: the query does not order by it, and the
caller never named it;
- a projection the library synthesized;
- `PolicySimulator`;
- `Explain`;
- schema building.
- Events buffer on the context (`PendingAuditEvents`); the query path never calls a sink.
- Drain once per request with `DwPolicy.DrainAuditAsync(context, sink)`. Undrained events are lost with the context.
- `Caps.MaxAuditEvents` bounds the undrained buffer.
- The event that does not fit refuses the query with `CapExceeded` (9), `SourceOrigin` containing `"MaxAuditEvents"`, even in dry run.
- Draining makes room again.
- A sink must throw when it fails to write. It is passed per call and never stored in options, so a scoped sink works.
- ASP.NET Core package: `app.UseDwPolicyAudit()` drains the context stored on `HttpContext.Features` to the `IDwAuditSink` registered in DI, after each response.
### Refused queries — `DwPolicyOptions.AuditRefusals` (3.1.0)
`[DwAudit]` records uses. A caller probing for columns they may not read is refused at every guess, so a log of uses never shows the probe; this records the refusals.
- Off by default. When true, every `PolicyException` raised by a `PolicyQueryable<T>` method, terminal or composable, and the `PolicyContextNotPrepared` refusal of `ApplyPolicy(context)`, is written to the caller's buffer (`PendingAuditEvents`) and drains to `IDwAuditSink` through `DwPolicy.DrainAuditAsync` or the ASP.NET Core audit middleware, like `[DwAudit]` events.
- It covers a refusal raised anywhere beneath the handle method: the gate, resolution, a store provider, a transform.
- The event:
- `EntityType` = the queried type's `FullName`; `Effect` = `Deny`; `DryRun` = false; `ErrorCode` = the refusal's code;
- `Feature` and `Tier` = the refusal's own, so a store provider's refusal reports `All` and `Strict`;
- `Subjects` and `Purpose` = the context's;
- `FieldPath` = the field the refusal was about, as its canonical path, in both tiers. A `FieldDeniedFor*`, `OperatorNotAllowed` or `RequiredFilterMissing` refusal, a `CapExceeded` refusal from `MaxNavigationDepth` or `MaxAuditEvents`, and `MissingContextValue`, which records the scoped field, are all recorded under the path although the refusal itself named the caller's alias or, under `Strict`, said `"*"`. A name that matches nothing (`Strict`) is recorded as the caller sent it, trimmed and with empty segments dropped. Every other refusal records its own `FieldPath` (section 30): `"*"` for a whole-request refusal such as `QueryStringDenied`, `QueryCostExceeded`, a cap on the request's size or `ApplyPolicy`'s `PolicyContextNotPrepared`; the entity's short type name for a store provider's refusal; the name as written for `AmbiguousFieldName`.
- The recorded path is cut to its first 256 characters followed by `…` (3.1.0). Then every character in Unicode category Control (Cc), Format (Cf), Line Separator (Zl) or Paragraph Separator (Zp) is written as `\u` and four lowercase hex digits: a line feed as `\u000a`, U+2028 as `\u2028`, U+202E as `\u202e`. A character outside the Basic Multilingual Plane is judged whole, and both halves of its surrogate pair are escaped. A name that matches nothing is text the caller wrote: a line break in it would forge a second entry in a log written one event per line, a format character such as U+202E would reverse the text after it without showing itself, and a name a megabyte long would be kept whole.
- Written at most once per refusal, from an exception filter: the refusal is never changed, caught or swallowed.
- A full buffer (`Caps.MaxAuditEvents`) records nothing, and the original refusal is still thrown.
- Not written:
- what dry run only records: it throws nothing there, so there is no refusal. A refusal dry run still throws (section 17) is written, with `DryRun` false;
- a refusal with no guarded context, such as `PolicyRequired` on an unguarded read of a `RequirePolicy` type;
- a `PolicySimulator` refusal, which runs on a copy of the context.
- Off by default because it changes what reaches a sink: a sink registered for `[DwAudit]` starts receiving events with an `ErrorCode`, and a deployment with no sink is warned by the ASP.NET Core audit middleware on every refused request.
---
## 23. Discovery: catalogue, schema, simulation
### DwEntityCatalog
```
DwEntityCatalog sealed class, DynamicWhere.ex.Policies.Discovery; DwPolicyOptions.Entities
DwEntityCatalog()
Expose<T>(string? name = null) -> DwEntityCatalog chainable
Expose(Type type, string? name = null) -> DwEntityCatalog
Resolve(string? name) -> Type? public name (case-insensitive), then exact Type.FullName; else null
NameOf(Type type) -> string? public name; null when never exposed
ToArray() -> Type[]
Entities : IReadOnlyDictionary<Type, string> read-only view
IsEmpty : bool
Freeze() -> void
```
- A null or blank `name` means `type.Name`; names are trimmed.
- Errors:
- a name already held by another type → `ArgumentException`;
- null type → `ArgumentNullException`;
- exposing after `Configure` → `InvalidOperationException`.
- Exposing a type again under another name keeps the old name resolvable, and `NameOf` returns the newest.
- Only exposed types can be described. `Resolve` gives the same null for a type that does not exist and one never exposed.
### PolicySchemaBuilder
```
PolicySchemaBuilder static class
Describe(Type entityType, DwEntityCatalog catalogue, DwPolicyContext context, DwPolicyOptions options,
PolicyResolver resolver, PolicySchemaRequest? request = null) -> PolicySchema
ResolveNavigation(Type entityType, string path, DwPolicyContext context,
DwPolicyOptions options, PolicyResolver resolver) -> string? canonical navigation path or null
PolicySchemaRequest sealed class; every member optional
Paths : IReadOnlyList<string>? { get; init; } navigation roots, canonical or alias per segment; null/empty = the entity
Depth : int? { get; init; } levels from each root; null = Caps.SchemaDepth; 1 = the root's own fields
```
```csharp
PolicySchema schema = PolicySchemaBuilder.Describe(
typeof(Employee), DwPolicy.Options.Entities, caller, DwPolicy.Options, DwPolicy.Resolver,
new PolicySchemaRequest { Paths = new[] { "Manager" }, Depth = 2 });
```
```
PolicySchema sealed class
Entity : string catalogue name
EntityType : string Type.FullName, what rules match on
Roots : IReadOnlyList<string> canonical roots; empty when rooted at the entity
Depth : int levels actually covered after clamping (largest over roots)
MaxDepth : int Caps.MaxNavigationDepth
Truncated : bool MaxSchemaFields cut Fields short
Fields : IReadOnlyList<PolicySchemaField> by Group (ungrouped last), Order (null last), Name; case-insensitive
Nodes : IReadOnlyList<PolicySchemaNode> by Path
PolicySchemaField sealed class
Path : string canonical path; what a rule names
Name : string alias, else Path; what a caller-facing UI shows
Parent : string? navigation path it hangs under; null for the entity's own fields
DataType : DataType
CanWhere CanSelect CanOrder CanGroup CanAggregate CanSegment : bool
Can(PolicyFeature feature) -> bool
IsMasked : bool Select effect is Mask: the value is transformed on output
AllowedOperators : IReadOnlyList<Operator>? null = unrestricted
AllowedValues : IReadOnlyList<string>?
IsRequiredInWhere : bool
CostWeight : int elected weight, else Caps.DefaultFieldCost
Label Description Group : string? Order : int?
PolicySchemaNode sealed class
Path : string
Name : string alias, else Path
Parent : string? null at the root of the view
Entity : string? catalogue name of the navigation's type; null when not exposed
Depth : int level of this node's own fields; the entity's fields are level 1
Expanded : bool the walk listed its fields
RemainingDepth : int navigation levels a request rooted here could still return; 0 = nothing to open
```
- A field is listed only when:
- its type maps to a `DataType`, and
- the caller may use it for at least one feature.
- A fully denied field (`[DwDenied]`) never appears. Audited fields are not marked.
- The entity is level 1; a root path of n segments is level n + 1. A root at level L walks to `min(L + Depth - 1, MaxNavigationDepth)`.
- A larger `Depth` is clamped, never refused; read `Depth` and `MaxDepth` back.
- `Depth < 1` → `ArgumentException`.
- A navigation is expanded while it is below that level and its type has appeared fewer than `SchemaCycleLimit` times on the path.
- The count restarts at each requested root, so `Manager.Manager` is described by requesting it as a root.
- Expanded nodes with nothing usable beneath them are removed. Unexpanded nodes are listed without checking beneath.
- Overlapping roots list each field and node once.
- Reaching `MaxSchemaFields` stops the walk and sets `Truncated`.
- `Describe` → `ArgumentException` when:
- the type is not exposed;
- a root names nothing;
- a root names a simple field the caller can see;
- a root has `>= MaxNavigationDepth` segments.
- A root naming a field the caller cannot use is answered as naming nothing.
- `ResolveNavigation` returns null for nothing, and throws `ArgumentException` for a blank path, a visible simple field, or a path too deep.
- A schema is resolved for the caller on every call and never cached. With a store configured, the context must be prepared.
### PolicySimulator
```
PolicySimulator static class
Simulate<T>(Filter filter, DwPolicyContext context, DwPolicyOptions options, PolicyResolver resolver) -> PolicySimulation<Filter>
Simulate<T>(Summary summary, DwPolicyContext context, DwPolicyOptions options, PolicyResolver resolver) -> PolicySimulation<Summary>
Simulate<T>(Segment segment, DwPolicyContext context, DwPolicyOptions options, PolicyResolver resolver) -> PolicySimulation<Segment>
Simulate<TClause>(Type entityType, TClause clause, DwPolicyContext context,
DwPolicyOptions options, PolicyResolver resolver) -> PolicySimulation<TClause>
where T : class; where TClause : class
PolicySimulation<TClause> where TClause : class sealed class
PolicySimulation(TClause? clause, PolicyTrace trace, PolicyException? refusal)
Clause : TClause? sanitized copy: drops applied, forced predicates injected, names canonical; null when refused
Trace : PolicyTrace
Refusal : PolicyException? null when it would run
WouldRun : bool Refusal is null
```
```csharp
PolicySimulation<Filter> sim = PolicySimulator.Simulate<Employee>(filter, caller, DwPolicy.Options, DwPolicy.Resolver);
if (!sim.WouldRun) logger.LogInformation("{Code}", sim.Refusal!.ErrorCode);
```
- It sanitizes only: no database, no transforms, and no audit events on `context`, because it runs on a copy. The clause passed in is not modified.
- A `PolicyException` becomes `Refusal`. Any other exception propagates, unwrapped even from the runtime overload.
- Under `Strict`, outside dry run, a path that matches nothing is therefore a `Refusal` with the clause's `FieldDeniedFor*` code and `FieldPath` `"*"`, not a `LogicException` (3.1.0). The `Trace` names it.
- A simulated `Filter` or `Segment` that sends no orders gets the type's `DefaultOrder` in `Clause.Orders`, less the fields the caller may not order by (3.1.0).
- A simulation has no source, so it reads T as a source it cannot see into (3.2.0): every denial beneath a member
counts, and with no `Selects` a synthesized `Clause.Selects` keeps only members holding a value. The guarded query
over a projected row keeps its assigned objects, one over an entity keeps its columns, owned and complex members
and asks only about denials whose value it loads, and one in memory keeps values only.
- Runtime overload errors:
- a value-type `entityType`, or a `TClause` other than `Filter`, `Summary` or `Segment` → `ArgumentException`;
- null entity, context or options → `ArgumentNullException`.
- Handle-level refusals are not simulated: `QueryStringDenied`, `TransformRequiresMaterialization`, `PolicyRequired`,
and the `PolicyContextNotPrepared` that `ApplyPolicy(context)` raises. The simulator never reads `IsPrepared`, so
without a store an unprepared context simulates as `WouldRun`; a store provider still refuses a context it never
prepared.
- A simulated Summary's `Clause` includes the floor's `__dwGroupSize` aggregate and its HAVING.
---
## 24. Token vaults
```
IDwTokenVault interface, DynamicWhere.ex.Policies.Tokens
string GetOrCreate(string scope, string value); stable token for scope + value
DwToken static class; helpers for vault implementations
New() -> string 32 lowercase hex characters from 16 RandomNumberGenerator bytes
KeyFor(string scope, string value) -> string "<scope>:<lowercase hex SHA-256 of the UTF-8 value>"
blank scope → ArgumentException; null value → ArgumentNullException
InMemoryTokenVault : IDwTokenVault sealed class; parameterless constructor
GetOrCreate(string scope, string value) -> string
Count : int mappings held
Clear() -> void forgets every mapping; tests only (reissues every token)
```
- `InMemoryTokenVault` is process-local, thread-safe and unbounded (no eviction).
- It is keyed by `DwToken.KeyFor`, so raw values are never keys.
- Tokens are lost on restart, and two instances or processes issue different tokens for one value.
- `RedisTokenVault` (Redis package) and `EfTokenVault` (EF Core package) keep tokens across restarts and instances.
- A vault is read on the query path, once per value per row, from many threads. A remote vault must cache in process.
- Implementation contract:
- thread-safe;
- return a stable, non-blank token, and store it before returning;
- never return the input value;
- throw when the store is unreachable.
- The interface has no reverse lookup.
- `InMemoryTokenVault`: a null `value` → `ArgumentNullException`; an empty string gets its own token.
- A tokenizing query while `DwPolicyOptions.TokenVault` is null → `MissingTokenVault` (22).
---
## 25. Dynamic rules
Rules a store supplies at runtime, merged with attributes by precedence level. Types live in `DynamicWhere.ex.Policies.Storage` (rule, stores, serializers), `DynamicWhere.ex.Policies.Resolution` (`StorePolicyProvider`) and `DynamicWhere.ex.Policies.DTOs` (`ForcedPredicate`, `FieldFacts`, transform stages).
### PolicyRule — immutable; every check runs in the constructor
```
new PolicyRule(
DwSubjectKind subjectKind, must be a defined member
string? subjectKey, required unless Global; trimmed; Global stores null
string entityType, Type.FullName; trimmed; must contain '.'
string fieldPath, a path or "*"; trimmed, segments trimmed, empty segments dropped
PolicyFeature features, no bit outside All
PolicyEffect effect, must be a defined member
int priority = 0, tiebreak within a level; higher wins
bool enabled = true, false: kept in the store, never applied
DateTimeOffset? validFrom = null, inclusive; null = already valid
DateTimeOffset? validTo = null, exclusive; null = never ends
string? purpose = null, trimmed; blank -> null
TransformStage? transform = null, one stage of the field's chain
IReadOnlyList<Operator>? allowedOperators = null, copied; null = says nothing
string? alias = null,
ForcedPredicate? forced = null,
IReadOnlyList<Operator>? requiredOperators = null, copied; null = no requirement, empty = nothing satisfies it
FieldFacts? facts = null,
Guid? id = null, null -> Guid.NewGuid()
string? createdBy = null, DateTimeOffset? createdAt = null,
string? updatedBy = null, DateTimeOffset? updatedAt = null)
```
Get-only properties: `Id SubjectKind SubjectKey Level EntityType FieldPath Features Effect Priority Enabled ValidFrom ValidTo Purpose Transform AllowedOperators Alias Forced RequiredOperators Facts CreatedBy CreatedAt UpdatedBy UpdatedAt IsBroad`
```
static string? NormalizeSubjectKey(string? key) Trim().ToLowerInvariant(); blank -> null
bool AppliesAt(DateTimeOffset now) (ValidFrom null or <= now) and (ValidTo null or now < ValidTo)
bool MatchesSubject(DwPolicyContext context) Global, or context holds SubjectKind:SubjectKey (OrdinalIgnoreCase)
bool MatchesPurpose(DwPolicyContext context) Purpose null, or equal to context.Purpose (OrdinalIgnoreCase)
PolicyFragment ToFragment() fragment at Level; source renders "Rule {Id} [{Describe()}]"
string Describe() "Global" or "Kind:Key"
string ToString() "{Describe()} {EntityType}.{FieldPath} {Features} => {Effect}"
bool IsBroad SubjectKind != User
```
The constructor refuses:
- `ArgumentOutOfRangeException` when `subjectKind` or `effect` is not a defined member.
- `ArgumentException` for any of these:
- `features` with an unknown bit.
- `features` None when the rule carries nothing (no alias, operator list, forced predicate, transform or facts).
- `features` None with any effect other than Allow.
- A blank `entityType`, or one with no '.'.
- A blank `fieldPath`, or one with no segment.
- A non-Global rule with no `subjectKey`.
- `validTo <= validFrom`.
- On `"*"` it also refuses an alias, `requiredOperators`, a `transform`, and `facts` with a label, description, group, order or allowedValues. A forced predicate, a cost weight and an audit are accepted on `"*"`.
Not checked by the constructor, by any store or by POST /rules. These are stored, then throw `ArgumentException` on the query path for every caller the rule applies to:
- An alias that is blank, contains '.', or is `"*"`.
- A rule with a real field path whose `forced` predicate names a different field.
The level comes from the subject kind and is never stored:
```
Global -> DynamicGlobal=5 Tenant -> DynamicTenant=4 Custom -> DynamicTenant=4
Role -> DynamicRole=3 User -> DynamicUser=2 no stored rule reaches SealedAttribute=1
```
How a rule applies, evaluated on every query:
- **Entity:** `EntityType` equals `Type.FullName`, case-insensitive.
- **Field:** an exact path, case-insensitive. `"*"` means every field of the type.
- There is no partial wildcard: `"Orders.*"` matches nothing.
- A rule on a navigation does not cover the fields beneath it.
- **Subject:** a Custom identity carries no dimension name, so every Custom subject with the same value matches.
- **Purpose:** a rule with a purpose never applies to a caller whose `Purpose` is null. Write a denial that must always hold without a purpose.
- **Validity window:** tested against UtcNow on each query, never at load.
- **Disabled rules** never reach a snapshot.
- **Transform stages** elect one winner per stage. A rule can add a stage on top of a sealed chain but cannot replace a sealed stage.
### Building rule parts in code
`ForcedPredicate` and `FieldFacts` are listed in section 21 and the transform stage classes in section 18.
`MutateStage` can be built in code, but the Redis and EF Core stores refuse to save it.
```csharp
var deny = new PolicyRule(DwSubjectKind.Role, "Support", typeof(Employee).FullName!, "Position",
PolicyFeature.Select | PolicyFeature.Order, PolicyEffect.Deny, priority: 10,
validTo: DateTimeOffset.UtcNow.AddDays(30));
// Carries a predicate and decides nothing: features None, effect Allow.
var scope = new PolicyRule(DwSubjectKind.Global, null, typeof(Employee).FullName!, "*",
PolicyFeature.None, PolicyEffect.Allow,
forced: ForcedPredicate.FromContext("TenantId", Operator.Equal, DataType.Number, "TenantId"));
// The caller's institution or none (3.1.0): injects (InstitutionId = x OR InstitutionId IS NULL).
var shared = new PolicyRule(DwSubjectKind.Global, null, typeof(Role).FullName!, "*",
PolicyFeature.None, PolicyEffect.Allow,
forced: ForcedPredicate.FromContext("InstitutionId", Operator.Equal, DataType.Number, "TenantId", allowNull: true));
```
### Rule document — `PolicyRuleDocument`
`PolicyRuleDocument` is the only JSON format for a whole rule. Redis stores the whole document; EF stores columns plus the `detail` object. POST /rules takes a different body, `RuleRequest`.
```
static string PolicyRuleDocument.ToJson(PolicyRule rule)
static PolicyRule PolicyRuleDocument.ToRule(string json) goes through the PolicyRule constructor
static string? PolicyRuleDocument.DetailToJson(PolicyRule rule) null when the rule has no detail
static RuleDetail PolicyRuleDocument.ReadDetail(string? json) blank -> RuleDetail.None
static TEnum PolicyRuleDocument.ToEnum<TEnum>(string? name, string what)
static PolicyFeature PolicyRuleDocument.ToFeatures(string? name) e.g. "Where, Select"
new RuleDetail(TransformStage? transform, IReadOnlyList<Operator>? allowedOperators, string? alias,
ForcedPredicate? forced, IReadOnlyList<Operator>? requiredOperators, FieldFacts? facts = null) RuleDetail.None
static TransformStage PolicyPayload.ToStage(string json) static string PolicyPayload.ToJson(TransformStage stage)
```
```jsonc
{
"id": "0b4c2f1e-7d0a-4c1e-9a55-2f6f5d0e9b10", // GUID; absent -> new id
"subjectKind": "Role", // required
"subjectKey": "Support", // absent for Global
"entityType": "MyApp.Models.Employee", // required
"fieldPath": "Email", // required
"features": "Select", // required; comma-separated flag names, "None" or "All"
"effect": "Mask", // required
"priority": 10, "enabled": true, // defaults 0 and true
"validFrom": "2026-09-01T00:00:00+00:00", // ISO 8601; validTo the same; absent = null
"purpose": "support",
"createdBy": "ops", "createdAt": "2026-09-01T00:00:00+00:00", "updatedBy": "ops", "updatedAt": "...",
"detail": { // absent when the rule has none of these
"transform": { "kind": "Mask", "allowAggregate": false, "minGroupSize": 0,
"strategy": "Email", "keepStart": 0, "keepEnd": 0, "maskChar": "*", "preserveLength": true },
"allowedOperators": ["Equal", "In"],
"requiredOperators": ["Equal"],
"alias": "ContactEmail",
"forced": { "fieldPath": "Email", "operator": "IsNotNull", "dataType": "Text" },
// a null check takes neither value nor contextValue; a comparison
// takes one of them, plus "allowNull": true to admit null (3.1.0)
"facts": { "label": "Email", "description": "...", "group": "Contact", "order": 1,
"allowedValues": ["a", "b"], "cost": 5, "audit": "Where, Select" }
}
}
```
Transform payload keys, read by `PolicyPayload.ToStage`:
```
every stage kind (required: Mask | Generalize | Format | Truncate | Default) allowAggregate false minGroupSize 0
Mask strategy (required) keepStart 0 keepEnd 0 maskChar "*" (exactly one char) preserveLength true
pattern replacement text tokenScope
Generalize mode (required) step 0 part "Year" decimals 0
Format format (required)
Truncate length (required) ellipsis
Default value; the key being present, even as null, means a value was supplied
```
Document rules:
- Property names must be exact camelCase. Unknown properties are ignored.
- Enums are names, matched case-insensitively. The following throw `ArgumentException`:
- A JSON number.
- A blank value, or a name that is not defined.
- A string of digits, for the rule-level enums: `subjectKind`, `effect`, `features`, operators, `dataType` and `audit`.
- `"kind": "Mutate"` is refused both when reading and when writing.
- `forced` holds either `value` or `contextValue`, or neither for IsNull/IsNotNull. Both together throw `ArgumentException`.
- A `contextValue` on IsNull/IsNotNull throws `ArgumentException` when the rule is read (3.1.0): the reader builds it
through `ForcedPredicate.FromContext`, which refuses a null check (section 21). Such a rule never worked: the key was
still required, and its value landed on a null check that validation refuses, so every guarded query on the type
failed.
- A `value` on IsNull/IsNotNull is still accepted and ignored.
- `forced.allowNull` (3.1.0): `true` injects `(field op value OR field IS NULL)`, as `[DwForceWhere(AllowNull = true)]` does.
- It is written only when true, so a document for any other predicate is the one earlier releases wrote and read. Absent or JSON null reads as false.
- Anything but a JSON boolean throws `ArgumentException` (the string `"true"` included).
- `true` on a null check (`IsNull` / `IsNotNull`) throws `ArgumentException`, whether or not the object also carries a `value` or `contextValue` (3.1.0). A null check ignores a constant, and an `IsNotNull` rule widened this way would render `(field IS NOT NULL OR field IS NULL)`, a scope that scopes nothing. Without `allowNull`, a stray `value` on a null check is still ignored; a `contextValue` there is refused (above).
- The document knows no member type, so `true` is not refused on a member that can never be null; there the rule injects the comparison alone (section 21).
- An absent operator list and an empty one stay distinct.
- A document or row that cannot be read is never skipped: it fails the load that reads it.
- A broad rule fails the load: fatal at startup, and a refresh failure afterwards, where `StoreFailure` applies.
- A `User` rule is read only when a context is prepared, so it fails `DwPolicy.PrepareAsync` for the callers it
names, in every `StoreFailure` mode, and does not degrade the provider.
---
## 26. Rule stores and StorePolicyProvider
### Contracts
```
interface IDwPolicyStore
ValueTask<StoreSnapshot> LoadAsync(CancellationToken ct) broad zone
ValueTask<NarrowZone> LoadNarrowAsync(IReadOnlyList<string> userIdentities, CancellationToken ct)
ValueTask<long> GetVersionAsync(CancellationToken ct)
IAsyncEnumerable<long>? WatchAsync(CancellationToken ct) null = cannot notify
interface IDwPolicyWritableStore : IDwPolicyStore
ValueTask<PolicyRule> UpsertAsync(PolicyRule rule, CancellationToken ct) insert or replace by Id; bumps the version
ValueTask DeleteAsync(Guid id, CancellationToken ct) bumps the version even when the id is absent
interface IDwPolicyRefresher
ValueTask<long> RefreshAsync(CancellationToken ct) implemented by StorePolicyProvider
```
- No store method runs on the query path. Stores are read at startup, by `PrepareAsync` and by the refresh loop.
- A read-only deployment registers only `IDwPolicyStore`.
- If `LoadAsync` or `LoadNarrowAsync` returns null, the provider throws `InvalidOperationException`.
- The shipped stores do four things a custom store should copy:
- Parse enums with `PolicyRuleDocument.ToEnum` and `ToFeatures`.
- Build user keys with `PolicyRule.NormalizeSubjectKey`.
- Call `SealedFields.Refuse` in `UpsertAsync`.
- Throw on a row that cannot be read.
### Zones
```
new StoreSnapshot(long version, DateTimeOffset loadedAt, IEnumerable<PolicyRule> rules)
long Version DateTimeOffset LoadedAt int Count static StoreSnapshot Empty IReadOnlyList<PolicyRule> For(Type entityType)
new NarrowZone(long version, IEnumerable<PolicyRule> rules)
long Version int Count static NarrowZone Empty IReadOnlyList<PolicyRule> For(Type entityType)
```
- **Broad zone (`StoreSnapshot`):** Global, Tenant, Role and Custom rules. It is loaded whole and swapped atomically.
- **Narrow zone (`NarrowZone`):** User rules. It is loaded when a context is prepared, for that caller's User identities only.
- **Wrong-zone rules:** a User rule in a snapshot, a non-User rule in a narrow zone, or a null throws `ArgumentException`, and the load fails.
- **Disabled rules** are dropped at construction, and `Count` counts only enabled rules. Validity windows are not applied here.
- **`For(Type)`** looks up `Type.FullName`, case-insensitive.
- **`StoreSnapshot.Empty`** (version 0) means no load has succeeded. An empty store returns a real snapshot instead.
- **`StoreSnapshot.LoadedAt`** is the store's own clock and decides nothing.
### InMemoryPolicyStore — core package
```
new InMemoryPolicyStore(Func<string, Type?>? resolveType = null) : IDwPolicyWritableStore, IDisposable
long Version starts at 0; +1 per write
InMemoryPolicyStore Seed(params PolicyRule[] rules) add or replace; one bump; null rule or sealed field -> ArgumentException
LoadAsync LoadNarrowAsync GetVersionAsync WatchAsync (yields every new version) UpsertAsync DeleteAsync
void Dispose() completes every watch
```
- It is thread-safe and not persisted. User identities match OrdinalIgnoreCase.
### Sealed-field refusal on write — `SealedFields`
```
static void SealedFields.Refuse(PolicyRule rule, Func<string, Type?>? resolveType, string parameterName)
```
- **Refusal:** throws `ArgumentException` when a SealedAttribute-level attribute fragment matches the rule's `FieldPath` and shares any of its features. A rule that names only features no sealed attribute covers is accepted.
- **Where it runs:**
- In `InMemoryPolicyStore.Seed` and `UpsertAsync`, `RedisPolicyStore.UpsertAsync` and `EfPolicyStore.UpsertAsync`, using the store's own `resolveType`.
- In POST /rules, using `DwPolicy.Options.Entities.Resolve`.
- **No resolver:** if `resolveType` is null or returns null, the rule is accepted. It still grants nothing, because a sealed attribute outranks every dynamic level when resolving.
- **Wildcards:** paths are compared exactly, so a `"*"` rule is not refused just because one field is sealed.
- **`DwEntityCatalog.Resolve(string? name)`** answers only for exposed types, by public name (case-insensitive) or exact `Type.FullName`.
- **Which catalogue:** take the delegate from the options you will configure, `options.Entities.Resolve`. Until `DwPolicy.Configure` runs, `DwPolicy.Options` is a separate default instance with an empty catalogue.
### StorePolicyProvider
```
: IDwPolicyProvider, IDwPolicyRefresher, IDisposable
static ValueTask<StorePolicyProvider> CreateAsync(IDwPolicyStore store, DwPolicyOptions options,
bool autoRefresh = true, CancellationToken ct = default)
ValueTask<DwPolicyContext> PrepareAsync(DwPolicyContext context, CancellationToken ct = default)
IReadOnlyList<PolicyFragment> GetFragments(Type entityType, DwPolicyContext context)
ValueTask<long> RefreshAsync(CancellationToken ct = default)
long Version bool IsDegraded DateTimeOffset LoadedAt TimeSpan Age Exception? LastError void Dispose()
```
- **`CreateAsync`** runs the first `LoadAsync` without catching it. Any failure throws, whatever `StoreFailure` is, and no provider is created.
- **Settings source:** `StoreFailure`, `MaxSnapshotAge` and `RefreshInterval` come from the `options` passed to `CreateAsync`, not from `DwPolicy.Options`. Pass the same instance to `DwPolicy.Configure`.
- **`DwPolicy.Configure(options, provider)`** always adds `AttributePolicyProvider` as well. `DwPolicy.StoreProviders` lists store providers in the order they were passed.
- **`DwPolicy.PrepareAsync(context, ct)`** calls each store provider's `PrepareAsync` in order, then sets `IsPrepared`. With no store configured it reads nothing, but still sets `IsPrepared`.
- **`PrepareAsync`:**
- Loads the narrow zone for the context's User identities, skipped when there are none.
- Pins onto the context the current snapshot, the provider's `LoadedAt`, the narrow zone, and the identities it read. Preparing again replaces the pin.
- A narrow-load failure propagates, in every mode, and so does a `User` rule the store cannot read.
- **`RefreshAsync`:**
- Always reloads and swaps atomically.
- Stamps `LoadedAt` from the provider's own UTC clock and clears `IsDegraded` and `LastError`.
- On failure it sets `IsDegraded` and `LastError`, keeps the last snapshot, and rethrows. `OperationCanceledException` is rethrown without degrading.
- **`Dispose()`** stops the background loop and waits up to 5 seconds.
`GetFragments` runs these checks in order on every guarded query:
```
1 this provider never prepared the context -> PolicyException PolicyContextNotPrepared
2 the context gained a User identity after it was prepared -> PolicyException PolicyContextNotPrepared
3 IsDegraded and StoreFailure = FailClosed -> PolicyException StoreUnavailable
IsDegraded and StoreFailure = StaticOnly -> returns no fragments from this store
4 now - pinned LoadedAt > MaxSnapshotAge (any mode, healthy too) -> PolicyException StoreUnavailable
5 one fragment for each rule in the pinned snapshot, then the pinned narrow zone, that applies now to this caller
```
- These refusals carry `FieldPath` = the entity's short type name and `Feature` = All.
- They report tier Strict, even under Convenience, and `SourceOrigin` = "{store type name}: {reason}".
- `IsDegraded` is read live, not pinned.
### Failure, staleness and refresh
```
StoreFailureMode LastKnownGood=0 FailClosed=1 StaticOnly=2
DwPolicyOptions StoreFailure = LastKnownGood MaxSnapshotAge = 00:15:00 RefreshInterval = 00:00:30
both TimeSpans must be > 0 (ArgumentOutOfRangeException); no value turns the ceiling off
```
```
Mode While IsDegraded When not degraded
LastKnownGood serves the pinned snapshot until the ceiling ceiling applies
FailClosed StoreUnavailable on every guarded query ceiling applies
StaticOnly no store fragments; ceiling not checked ceiling applies
```
- **Startup:** a failed first load throws in all three modes.
- **Background loop (`autoRefresh: true`):**
- It subscribes to `WatchAsync` when that returns non-null. If opening the watch throws, it polls only.
- It always polls `GetVersionAsync` every `RefreshInterval`. The first poll comes one interval after start.
- It calls `RefreshAsync` only when the reported version differs from the version being served. A lower version also counts as different.
- A poll whose version matches renews the snapshot without reloading: it stamps the load time (as of when it asked) and clears the degraded flag — unless a refresh or poll failed while its read was in flight, in which case it reloads instead (3.1.0).
- **Degraded state:**
- A failed poll sets `IsDegraded` and leaves `LastError` alone. A failed reload sets both.
- Cleared by a successful `RefreshAsync`, and by a poll that reads back the version being served.
- So a store that comes back is served again from the next poll, with or without a write to it.
- **The ceiling:**
- `LoadedAt` moves on `CreateAsync`, on a successful `RefreshAsync`, and on a poll that confirms the served version.
- With `autoRefresh: true` an unchanged healthy store keeps renewing through the poll, so the ceiling bites only when the store cannot be read.
- **With `autoRefresh: false` nothing renews it.** Such a host calls `RefreshAsync` itself, more often than `MaxSnapshotAge`, or every guarded query is refused once the ceiling passes.
- The ceiling measures from the provider load that a context pinned. A context kept longer than `MaxSnapshotAge` is refused even after the provider refreshes, so prepare one context per request.
- **Store-only denials:** under StaticOnly, a denial held only in the store is not applied while degraded.
- **Unreachable store:** `PrepareAsync` for a caller with a User identity throws, in every mode.
### Wiring a store
```csharp
var options = new DwPolicyOptions(); // or new DwPolicyOptions().Bind(configuration.GetSection("DynamicWhere:Policies"))
options.Entities.Expose<Employee>("Employee");
var store = new InMemoryPolicyStore(options.Entities.Resolve);
var provider = await StorePolicyProvider.CreateAsync(store, options);
DwPolicy.Configure(options, provider); // once; freezes options
builder.Services.AddSingleton<IDwPolicyStore>(store); // admin API reads
builder.Services.AddSingleton<IDwPolicyWritableStore>(store); // admin API writes
// once per request
DwPolicyContext caller = await DwPolicy.PrepareAsync(new DwPolicyContext().WithSubject(DwSubjectKind.User, userId));
```
- `AddDwPolicies(IConfiguration section, Action<DwPolicyOptions>? configure = null, params IDwPolicyProvider[] providers)` also accepts the provider. It builds its own options instance, and the provider still uses the one passed to `CreateAsync`.
- The companion packages ship no `IServiceCollection` extension. Construct stores, vaults and providers yourself, as above.
---
## 27. Redis package — DynamicWhere.ex.Policies.Redis
```
net6.0 dependencies: DynamicWhere.ex 3.1.0, StackExchange.Redis 2.8.24 namespace DynamicWhere.ex.Policies.Redis
new RedisPolicyStore(IConnectionMultiplexer redis, string? prefix = null, Func<string, Type?>? resolveType = null)
: IDwPolicyWritableStore
new RedisTokenVault(IConnectionMultiplexer redis, string? prefix = null) : IDwTokenVault
string GetOrCreate(string scope, string value) int CachedCount void ClearCache()
null redis -> ArgumentNullException; the multiplexer is not owned and never disposed
```
Key layout, from the internal `RedisPolicyKeys` (prefix trimmed; blank -> `dw:policy`):
```
{prefix}:rules hash ruleId -> rule document JSON broad rules
{prefix}:user:{key} hash ruleId -> rule document JSON key = NormalizeSubjectKey(subjectKey)
{prefix}:owner hash ruleId -> key of the hash holding the rule
{prefix}:version string counter; absent reads as 0
{prefix}:version pub/sub channel; message = the new version
{prefix}:tokens hash "{scope}:{sha256(value) lowercase hex}" -> 32 lowercase hex token
```
RedisPolicyStore:
- **Upsert:** `SealedFields.Refuse` first. Then one MULTI/EXEC transaction:
- deletes the rule from the hash named in `owner`, if that hash differs,
- HSETs the document and the owner entry,
- INCRs the version.
- **Upsert failures:** an EXEC that does not commit throws `InvalidOperationException`. After commit it PUBLISHes the version; a `RedisException` from that publish is swallowed and the poll catches up.
- **Delete:** one transaction removes the rule and its owner entry and always INCRs the version, then PUBLISHes.
- **LoadAsync:** reads the version before the rules. A document it cannot read throws `ArgumentException`.
- **LoadNarrowAsync:** one HGETALL per identity, skipping blank identities. Keys are lower-case, so casing never splits one user into two.
- **WatchAsync:** subscribes to the channel. The provider still polls `{prefix}:version`.
- **Shared Redis:** two applications on one Redis need different prefixes, for rules and for tokens.
RedisTokenVault:
- **Cache:** every mapping it resolves is cached in the process, unbounded. Tokens never change, so the cache cannot go stale.
- **Minting:** HSETNX, then HGET the winner after a lost race, so instances agree on one token.
- A field that disappears between the two calls throws `InvalidOperationException`.
- A `RedisException` propagates.
- **Query path:** Redis calls are synchronous and happen on a cache miss, during the query.
- **Protection:** the hash stores the scope in clear and an unkeyed SHA-256 of the value, never the value itself. Anyone who can read it can confirm a value they can guess, so guard it like the column it protects.
```csharp
IConnectionMultiplexer redis = await ConnectionMultiplexer.ConnectAsync(connectionString);
var options = new DwPolicyOptions { TokenVault = new RedisTokenVault(redis, "myapp:policy") };
options.Entities.Expose<Employee>("Employee");
var store = new RedisPolicyStore(redis, "myapp:policy", options.Entities.Resolve);
DwPolicy.Configure(options, await StorePolicyProvider.CreateAsync(store, options));
```
---
## 28. Entity Framework Core package — DynamicWhere.ex.Policies.EntityFrameworkCore
```
net6.0 dependencies: DynamicWhere.ex 3.1.0, Microsoft.EntityFrameworkCore.Relational 6.0.22
namespace DynamicWhere.ex.Policies.EntityFrameworkCore no raw SQL; any EF Core relational provider; ships no migrations
new EfPolicyStore(Func<DbContext> contexts, Func<string, Type?>? resolveType = null) : IDwPolicyWritableStore
new EfTokenVault(Func<DbContext> contexts) : IDwTokenVault
string GetOrCreate(string scope, string value) int CachedCount void ClearCache()
null contexts -> ArgumentNullException
class DwPolicyDbContext(DbContextOptions<DwPolicyDbContext> options) : DbContext
DbSet<DwPolicyRuleRecord> PolicyRules DbSet<DwPolicyVersionRecord> PolicyVersion DbSet<DwPolicyTokenRecord> PolicyTokens
OnModelCreating applies all three configurations below
sealed DwPolicyRuleConfiguration : IEntityTypeConfiguration<DwPolicyRuleRecord> const string Table = "DwPolicyRules"
sealed DwPolicyVersionConfiguration : IEntityTypeConfiguration<DwPolicyVersionRecord> const string Table = "DwPolicyVersion" const int SingleRowId = 1
sealed DwPolicyTokenConfiguration : IEntityTypeConfiguration<DwPolicyTokenRecord> const string Table = "DwPolicyTokens"
```
All of these are in `DwPolicyConfigurations.cs`; no type is named `DwPolicyConfigurations`.
```
DwPolicyRules Id Guid PK, never generated
SubjectKind string(32) required SubjectKey string(256) SubjectKeyNormalized string(256)
EntityType string(512) required FieldPath string(512) required
Features string(128) required Effect string(32) required
Priority int Enabled bool ValidFrom, ValidTo DateTimeOffset? (written as UTC) Purpose string(128)
Detail string, unbounded: JSON of transform, operator lists, alias, forced, facts; null when none
CreatedBy string(256) CreatedAt DateTimeOffset? (UTC) UpdatedBy string(256) UpdatedAt DateTimeOffset? (UTC)
index (SubjectKind, SubjectKeyNormalized, EntityType) index (EntityType, FieldPath)
DwPolicyVersion Id int PK, never generated (one row, Id = 1) Version long, concurrency token UpdatedAt DateTimeOffset
DwPolicyTokens Key string(512) PK, never generated = "{scope}:{sha256 hex}" Scope string(256) required, indexed
Token string(64) required CreatedAt DateTimeOffset (written as UTC)
DwPolicyRuleRecord Guid Id string SubjectKind string? SubjectKey string? SubjectKeyNormalized string EntityType
string FieldPath string Features string Effect int Priority bool Enabled
DateTimeOffset? ValidFrom DateTimeOffset? ValidTo string? Purpose string? Detail
string? CreatedBy DateTimeOffset? CreatedAt string? UpdatedBy DateTimeOffset? UpdatedAt
static string? Normalize(string? key) static DwPolicyRuleRecord FromRule(PolicyRule rule) PolicyRule ToRule()
DwPolicyVersionRecord int Id long Version DateTimeOffset UpdatedAt
DwPolicyTokenRecord string Key string Scope string Token DateTimeOffset CreatedAt
```
- Enums are stored as names.
- `ToRule()` goes through the `PolicyRule` constructor, so a bad row throws `ArgumentException` and the load fails.
- `SubjectKey` keeps the key as typed. The narrow load matches on `SubjectKeyNormalized` (lower-case), so database collation never matters.
Your own context:
```csharp
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.ApplyConfiguration(new DwPolicyRuleConfiguration());
modelBuilder.ApplyConfiguration(new DwPolicyVersionConfiguration());
modelBuilder.ApplyConfiguration(new DwPolicyTokenConfiguration()); // only needed for EfTokenVault
}
// generate the migration in your own project, for your provider
var store = new EfPolicyStore(() => new AppDbContext(appDbOptions), options.Entities.Resolve);
```
The packaged `DwPolicyDbContext` is declared in the package assembly. Set `MigrationsAssembly` to your project everywhere its options are built: the runtime options and any `IDesignTimeDbContextFactory<DwPolicyDbContext>`.
```csharp
var policyDb = new DbContextOptionsBuilder<DwPolicyDbContext>()
.UseNpgsql(connection, sql => sql.MigrationsAssembly("YourProject"))
.Options;
var store = new EfPolicyStore(() => new DwPolicyDbContext(policyDb), options.Entities.Resolve);
DwPolicy.Configure(options, await StorePolicyProvider.CreateAsync(store, options));
```
EfPolicyStore:
- **Contexts:** `contexts` is called once per operation and the store disposes what it returns. Return a new context every time, never a shared one. The model must include the rule and version configurations.
- **Change detection:** `WatchAsync` returns null, so changes arrive only by polling the version row every `RefreshInterval`.
- **Writes:**
- Upsert and delete apply the row change and the version bump in one `SaveChanges`. The version row is created at 1 if missing.
- A `DbUpdateException` is retried on a fresh context, up to 8 attempts, with a delay of 2–11 ms × attempt number, then rethrown.
- Upsert overwrites every column of the row with the same Id. Delete bumps the version even when no row matched.
- **LoadAsync:** reads the version first. The broad load takes rows where SubjectKind ≠ "User", SubjectKind is null, or SubjectKeyNormalized is null, so a malformed User row fails the load instead of disappearing.
EfTokenVault:
- **Contexts:** `contexts` is called only on a cache miss and the context is disposed. The model must include `DwPolicyTokenConfiguration`.
- **Minting:** reads with no tracking, then inserts. On a `DbUpdateException` from a primary-key race it clears the change tracker and reads the winning row. If that row is still missing it throws `InvalidOperationException`.
- **Behaviour:** EF calls are synchronous. It caches in the process and uses the same scope-plus-digest keys as `RedisTokenVault`.
---
## 29. ASP.NET Core package — DynamicWhere.ex.Policies.AspNetCore
```
net6.0 dependencies: DynamicWhere.ex 3.1.0, FrameworkReference Microsoft.AspNetCore.App
namespace DynamicWhere.ex.Policies.AspNetCore
```
### MapDwPolicyAdmin
```
static IEndpointRouteBuilder MapDwPolicyAdmin(this IEndpointRouteBuilder endpoints,
Action<DwPolicyAdminOptions>? configure = null)
DwPolicyAdminOptions
string RoutePrefix = "/dw-policies" trailing '/' trimmed
string? ReadPolicy authorization policy: schema, GET rules, explain, simulate, health
string? WritePolicy authorization policy: POST rules, DELETE rules
bool AllowAnonymousAccess = false true: all seven routes, writes included, have no authorization
DwClaimsOptions Claims { get; } builds the caller for schema, explain and simulate
```
- **Validation** happens at the call, before any route is added:
- A blank `RoutePrefix` throws `InvalidOperationException`.
- A blank `ReadPolicy` or `WritePolicy` throws `InvalidOperationException` unless `AllowAnonymousAccess = true`. So calling it with no `configure` throws.
- A null `endpoints` throws `ArgumentNullException`.
- **Authorization:** the host registers both policy names with `AddAuthorization` and runs authentication and authorization middleware. A caller that fails a policy gets 403.
- **Routes** are mapped one by one with the prefix concatenated; there is no `MapGroup`. It returns the builder, not a convention builder.
- **DI, per request:**
- GET /rules needs `IDwPolicyStore`; POST and DELETE need `IDwPolicyWritableStore`. A missing registration gives 501.
- Register the instance your provider reads under both types.
- **Global state:** everything else comes from the static `DwPolicy` (`Options.Entities`, `Resolver`, `StoreProviders`). Only exposed entities can be asked about.
- **JSON:**
- Handler error bodies are `{ "error": "message" }`, and property names are camelCase.
- The library registers no JSON enum converter. Enum members inside schema fields and a `Filter` follow the host's minimal-API JSON options; with default options the test suite posts `Filter` enums as numbers.
- **The caller:** schema, explain and simulate build the caller from the request principal via `http.GetPolicyContextAsync(options.Claims)`. There is no parameter to impersonate another caller.
- With `Claims.AllowAnonymous = false`, an unauthenticated principal throws `InvalidOperationException` there, unhandled.
- A host that authorizes without authenticating must set `Claims.AllowAnonymous = true`.
### Endpoints
```
Method Route Policy Body Success Handler statuses
POST {prefix}/schema Read SchemaRequest 200 schema 400, 404
GET {prefix}/rules?subject= Read — 200 [rule] 501
POST {prefix}/rules Write RuleRequest 200 rule 400, 501
DELETE {prefix}/rules/{id:guid} Write — 204 501
POST {prefix}/explain Read ExplainRequest 200 [explanation] 400, 404
POST {prefix}/simulate Read SimulateRequest 200 simulation 400, 404
GET {prefix}/health Read — 200 health 503 (same body)
```
- An unmapped verb gives 405, a non-GUID id gives 404 (route constraint), and a body that is not JSON gives 400.
- These propagate unhandled:
- the anonymous-principal `InvalidOperationException`;
- a `PolicyException` such as `StoreUnavailable` from schema or explain (`PolicyException` derives from `LogicException`, not `ArgumentException`);
- exceptions from the store's own `UpsertAsync` or `DeleteAsync`, and store I/O errors.
```
SchemaRequest(string? Entity, IReadOnlyList<string>? Paths = null, int? Depth = null)
ExplainRequest(string? Entity, string? Field, IReadOnlyList<string>? Paths = null, int? Depth = null)
SimulateRequest(string? Entity, Filter? Filter)
RuleRequest(Guid? Id, string? SubjectKind, string? SubjectKey, string? EntityType, string? FieldPath,
string? Features, string? Effect, int Priority = 0, bool Enabled = true,
DateTimeOffset? ValidFrom = null, DateTimeOffset? ValidTo = null,
string? Purpose = null, string? Alias = null)
```
### POST /schema — `{ entity, paths?, depth? }`
- **`entity`:** an exposed type's public name (case-insensitive) or its exact `Type.FullName`. An unknown type and an unexposed type both give 404 `No entity named '{entity}'.`
- **`paths`:** navigation roots, as canonical names or aliases; several are allowed per request. It is a body list rather than a query string because a comma is legal inside an alias.
- **`depth`:** levels to walk from each root. Null means `Caps.SchemaDepth`. A value above the cap is clamped, not refused.
- **400:**
- `depth` below 1,
- a blank path,
- a path with at least `MaxNavigationDepth` segments,
- a path naming a value field the caller can see.
- **404:** a path naming nothing, or a value field the caller cannot see: `'{path}' is not a navigation of '{entity}'.`
```
{ entity, entityType, roots[], depth, maxDepth, truncated,
fields[ { path, name, parent, dataType, canWhere, canSelect, canOrder, canGroup, canAggregate, canSegment,
isMasked, allowedOperators, allowedValues, isRequiredInWhere, costWeight, label, description, group, order } ],
nodes[ { path, name, parent, entity, depth, expanded, remainingDepth } ] }
```
- **Top-level fields:**
- `depth` = the levels actually covered.
- `maxDepth` = `Caps.MaxNavigationDepth`.
- `truncated` = `MaxSchemaFields` cut the list short.
- **Field entries:** `name` = the alias, or the path. `parent` = the owning node's path, or null.
- **Node entries:**
- `expanded` = its fields were described.
- `remainingDepth` = further levels beneath it. 0 means nothing to open, and the value accounts for `SchemaCycleLimit`.
- `entity` = the catalogue name, or null.
- A sealed-denied field, such as one marked `[DwDenied]`, is absent.
### GET /rules?subject=Kind[:Key]
- **Source:** reads the store directly (`LoadAsync`, or `LoadNarrowAsync([Key])` for `User`), not the provider's snapshot, so a write shows up at once.
- **What it lists:** enabled rules only, for exposed entity types only, in no order. Validity windows are not applied.
- **No `subject`:** every enabled broad rule. User rules need `subject=User:{key}`; `subject=User` alone returns `[]`.
- **`Kind` alone:** every rule of that kind. A key is compared after `NormalizeSubjectKey`. A kind that does not parse is treated as Custom.
```
rule = { id, subjectKind, subjectKey, level, entityType, fieldPath, features, effect, priority, enabled,
validFrom, validTo, purpose, alias, allowedOperators, requiredOperators,
facts: { label, description, group, order, allowedValues, cost, audit } | null,
createdBy, createdAt, updatedBy, updatedAt }
```
- Enums appear as names and `subjectKey` as typed. `transform` and `forced` are not included.
### POST /rules — `RuleRequest`
```json
{ "id": null, "subjectKind": "Role", "subjectKey": "Support", "entityType": "MyApp.Models.Employee",
"fieldPath": "Position", "features": "Select, Order", "effect": "Deny",
"priority": 10, "enabled": true, "validFrom": null, "validTo": "2026-12-31T00:00:00Z",
"purpose": null, "alias": null }
```
- **Enums** are names: `subjectKind` is Global|Tenant|Role|User|Custom, `features` is comma-separated flag names, `effect` is Allow|Mask|Deny. Digits, blanks or unknown names give 400.
- **`id`:** null creates a rule; an existing id replaces the whole rule.
- **What the body cannot carry:** a transform, operator lists, a forced predicate or facts. Write those with `IDwPolicyWritableStore.UpsertAsync(new PolicyRule(...))`.
- **Audit stamps:**
- `createdBy` and `updatedBy` = `Identity.Name`, else the `ClaimTypes.NameIdentifier` claim, else `sub`.
- `createdAt` and `updatedAt` = now.
- All four are stamped on every POST, a replace included, and any values in the body are ignored.
- **400 `{error}`:** any `PolicyRule` refusal (`ArgumentException`, `FormatException`, `OverflowException`), or a sealed field. The sealed check runs only when `entityType` resolves through `DwPolicy.Options.Entities`.
- **200:** the stored rule, in the GET shape.
- **Refresh:** it does not refresh the provider. Enforcement sees the rule after the next watch message or version poll, or after a `RefreshAsync` call.
### DELETE /rules/{id}
- Returns 204 whether or not the rule existed, and 501 when no `IDwPolicyWritableStore` is registered.
### POST /explain — `{ entity, field?, paths?, depth? }`
- **`entity`, `paths`, `depth`:** same handling and same 400/404 as /schema.
- **With `field`:** matched against this caller's schema fields by `path` or `name`, case-insensitive, and returns a one-element array.
- A field missing from that schema (misspelt, beyond depth, or denied and omitted) gives 404 `'{field}' is not a field of '{entity}' that this caller can use.`
- **Without `field`:** one entry per schema field.
```
[ { field, entityType, name, isSealed,
features[ { feature, effect, decidedBy, level, attributionAmbiguous, tiedWith[], overrode[] } ] } ]
```
- **`features`:** 6 entries, one per feature. When nothing spoke, `effect` is Allow and `decidedBy` and `level` are null.
- **`isSealed`:** an attribute decided at least one feature absolutely.
- **Sources** render as `Rule {id} [Global|Kind:Key]`, or as the attribute's name with ` (sealed)` appended when it is sealed.
- **`attributionAmbiguous`:** `tiedWith` is non-empty, so `decidedBy` names one of several equal sources.
### POST /simulate — `{ entity, filter }`
- **Errors:**
- 404 for an unknown entity.
- 400 `A filter is required.`
- 400 `{error}` when core validation rejects the filter (a `LogicException`, for example a select naming a property the type lacks).
- Under the Strict tier, outside dry run, a name the type lacks is a policy refusal instead (3.1.0): 200 with `wouldRun: false`, `refusal.field` `"*"`, `refusal.reason` null, and the trace naming it.
- **Strict refusals (3.1.0):** `refusal.field` is `"*"` for codes 1–6 and `CapExceeded`, and `refusal.reason` is null for codes 1–6, as on the `PolicyException` (section 30). The `trace` still names the field and the reason.
- **No side effects:** nothing executes and nothing is audited; it runs on a copy of the caller's context.
```
{ wouldRun, filter: Filter | null,
refusal: { code, field, feature, reason } | null, code = PolicyErrorCode name; reason = SourceOrigin
trace[ { field, feature, action, reason } ] }
```
- **A policy refusal** is 200 with `wouldRun: false` and `filter: null`.
- **Otherwise** `filter` is the sanitized copy: denied parts dropped, predicates injected, names made canonical, and, when it sent no orders, the type's `DefaultOrder` added (3.1.0).
### GET /health
```
{ healthy, configured, tier, dryRun, maxSnapshotAge,
stores[ { version, loadedAt, ageSeconds, degraded, lastError } ] } lastError = exception message, or null
```
- `healthy` = no store provider is configured, or none is degraded. It returns 200 when healthy and 503 with the same body when not.
- An `ageSeconds` above `maxSnapshotAge` does not make it unhealthy.
### ClaimsPrincipal to context
```
static class DwClaimsAdapter
static ValueTask<DwPolicyContext> CreateContextAsync(ClaimsPrincipal principal, DwClaimsOptions options, CancellationToken ct = default)
static DwPolicyContext FromClaims(ClaimsPrincipal principal, DwClaimsOptions options) not prepared
static ValueTask<DwPolicyContext> ToPolicyContextAsync(this ClaimsPrincipal principal, DwClaimsOptions options,
CancellationToken ct = default) ClaimsPrincipalPolicyExtensions
static ValueTask<DwPolicyContext> GetPolicyContextAsync(this HttpContext http, DwClaimsOptions options) DwPolicyHttpContextExtensions
DwClaimsOptions
IList<string> UserClaimTypes = { ClaimTypes.NameIdentifier, "sub" }
IList<string> RoleClaimTypes = { ClaimTypes.Role, "role", "roles" }
IList<string> TenantClaimTypes = { "tenant", "tenant_id", "tid" }
IDictionary<string, DwSubjectKind> SubjectClaimTypes claim type -> kind; empty; keys case-insensitive
IDictionary<string, string> ValueClaimTypes context key -> claim type; empty; keys case-insensitive
bool AllowAnonymous = false
string? Purpose = null copied to DwPolicyContext.Purpose
```
- **Two builders:** `CreateContextAsync` is `FromClaims` followed by `DwPolicy.PrepareAsync`. A `FromClaims` context is unprepared, and since 3.1.0 `ApplyPolicy(context)` refuses it with `PolicyContextNotPrepared` whether or not a store is configured: prepare it before querying, or use `CreateContextAsync`. `FromClaims` is for adding to the context before preparing it.
- **Refusals:**
- A null principal or options throws `ArgumentNullException`.
- If `Identity.IsAuthenticated` is not true and `AllowAnonymous` is false, it throws `InvalidOperationException`.
- With `AllowAnonymous = true`, an unauthenticated principal's claims are still read.
- **Subjects:** every claim of every listed type becomes a subject, so a caller keeps all their roles, not just the first. Blank values are skipped and duplicates (case-insensitive) collapse.
- **`ValueClaimTypes`:** the first claim of the mapped type becomes `WithValue(key, value)`. A missing claim leaves the key absent, and a forced predicate that reads it refuses with `MissingContextValue`.
- **`GetPolicyContextAsync`:**
- Builds the context from `http.User`, prepares it with `http.RequestAborted`, and stores it with `http.Features.Set`.
- Later calls in the same request return that same instance and ignore `options`.
### Audit middleware
```
app.UseDwPolicyAudit(); static IApplicationBuilder UseDwPolicyAudit(this IApplicationBuilder app)
new DwPolicyAuditMiddleware(RequestDelegate next, ILogger<DwPolicyAuditMiddleware>? log = null) Task InvokeAsync(HttpContext http)
```
- **When it runs:** after the rest of the pipeline, in a `finally`, so also when the request threw. It then drains the request's pending audit events.
- **Which context:** only the `DwPolicyContext` in `http.Features`, which `GetPolicyContextAsync` stores there. For a context you built yourself, call `http.Features.Set(context)`. With no context or no pending events it does nothing.
- **Sink:** `IDwAuditSink`, resolved from `RequestServices` (so a scoped sink works), written through `DwPolicy.DrainAuditAsync`.
- **Failures:**
- No sink registered: logs a warning and discards the events. The warning ends "Register one, or stop recording them: remove [DwAudit] from the fields that produced them, or turn off DwPolicyOptions.AuditRefusals." (3.1.0).
- A throwing sink: logs an error and does not rethrow. The unwritten events stay on the context and are lost with the request.
- The request's own exception is never replaced.
- **Events:** the middleware records nothing itself. Events come from audited fields (`[DwAudit]`, or a rule's `facts.audit`), one `DwAuditEvent` per use, and, with `DwPolicyOptions.AuditRefusals` (3.1.0), one per refused guarded query, with `OccurredAt EntityType FieldPath Feature Effect Subjects Purpose Tier DryRun ErrorCode`.
- **Placement:** register it before anything that touches audited fields. The source recommends placing it before authentication and routing.
---
## 30. Policy error codes
```
PolicyException : LogicException namespace DynamicWhere.ex.Exceptions
PolicyException(PolicyErrorCode errorCode, string fieldPath, PolicyFeature feature, DwTier tier)
ErrorCode : PolicyErrorCode branch on this
Code : string ErrorCode.ToString(), for logs and JSON
FieldPath : string the name the caller wrote; "*" for the whole request, including the
PolicyContextNotPrepared ApplyPolicy raises; under the Strict tier "*" for
codes 1–6, CapExceeded and MissingContextValue too (3.1.0); the entity's
short type name for PolicyRequired, StoreUnavailable and a store provider's
PolicyContextNotPrepared;
the transformed paths joined by ", " for TransformRequiresMaterialization and
AmbiguousGroupKey
Feature : PolicyFeature None for ApplyPolicy's PolicyContextNotPrepared; All for a store provider's refusals
Tier : DwTier the tier in force; always Strict for PolicyRequired, StoreUnavailable and a store
provider's PolicyContextNotPrepared (ApplyPolicy's carries the configured tier)
RuleId : string? { init; } the rule that decided, when exactly one source decided; null for codes 1–6
under the Strict tier (3.1.0)
SourceOrigin : string? { init; } the attribute, rule, cap or reason that decided, e.g.
"MaxPageSize cap (1000), request had 1001"; null for codes 1–6 and
MissingContextValue under the Strict tier (3.1.0)
Message "<Code>: field '<FieldPath>', feature '<Feature>', tier '<Tier>'."
```
`PolicyErrorCode` (namespace `DynamicWhere.ex.Policies.Enums`) numbers are fixed; new members are only appended.
A refusal throws from the method called on the guarded handle; the exception carries no trace (run the request
through `PolicySimulator` or in dry run to see the decisions).
```
# Name Raised when In dry run
1 FieldDeniedForWhere a Where, Having or Segment condition uses a field denied for Where recorded
(both tiers; under Strict a Segment condition gets code 6 instead)
2 FieldDeniedForSelect Strict: a selected field, or a navigation with a denied field beneath recorded
it, is denied for Select; any tier: "a.b" when its key "a.Id" is denied
3 FieldDeniedForOrder Strict: an order field is denied for Order (Convenience drops it) recorded
4 FieldDeniedForGroup a group field is denied for Group (both tiers) recorded
5 FieldDeniedForAggregate an aggregate field is denied, or is transformed without AllowAggregate recorded
on every stage (both tiers)
6 FieldDeniedForSegment a field denied for Segment is used in a Segment (both tiers); Strict: recorded
a Segment condition on a field the caller may not Select, and (3.1.0)
every other field refusal inside a Segment, codes 1–5 included
7 AllSelectsDenied Convenience dropped every requested select (FieldPath "*") recorded
8 OperatorNotAllowed an operator outside the field's allowed set ([DwOperators], rules) recorded
9 CapExceeded MaxPageSize, MaxConditions, MaxConditionDepth, MaxConditionSets, recorded, except the
MaxConditionValues, MaxAggregates, MaxOrderFields or MaxNavigationDepth audit buffer: throws
exceeded; or the context already holds MaxAuditEvents undrained events
10 PolicyRequired a DynamicWhere extension method called on a throws
[DwEntity(RequirePolicy = true)] type outside ApplyPolicy
11 RequiredFilterMissing a [DwRequireWhere] field has no AND-reachable condition using one of recorded
its operators
12 MissingContextValue a [DwForceWhere] ContextValue key is absent or null in the context; recorded
under Strict FieldPath "*" and no SourceOrigin (3.1.0)
13 AmbiguousFieldName a name could mean two fields: an alias equal to a path or another alias throws
14 QueryStringDenied Strict tier and getQueryString: true (FieldPath "*") recorded
15 AmbiguousGroupKey two summary rows share their key values once transformed throws
16 TransformRequiresMaterialization SelectDynamic, FilterDynamic, Group or Summary on the guarded handle throws
for a type with any transform for this caller
17 StoreUnavailable a FailClosed store provider is degraded, or the context's pinned throws
snapshot is older than MaxSnapshotAge
18 PolicyContextNotPrepared ApplyPolicy(ctx) sees a context that never went through PrepareAsync throws
(3.1.0, store or no store), a store provider sees one it never prepared,
or one that gained a User subject after it was prepared
19 QueryCostExceeded the request's total field cost is above MaxQueryCost (FieldPath "*"); recorded
under Strict checked after the field gates (3.1.0)
20 GroupTooSmall the caller used the reserved name "__dwGroupSize" as an alias, Having throws
field or order field. A small group never raises it: it is suppressed
21 MissingHashSalt MaskStrategy.Hash while DwPolicyOptions.HashSalt is empty throws
22 MissingTokenVault MaskStrategy.Tokenize while DwPolicyOptions.TokenVault is null throws
```
"recorded" means dry run writes a `Denied` decision to the trace and runs the query anyway.
Under the Strict tier (3.1.0) codes 1–6 name no field and no source, so a denied field, an alias and a name that
matches nothing on `T` answer with the same exception and the same message. Outside dry run, a condition, select,
order, group or aggregate path that matches nothing is refused with the code of its clause, as a `[DwDenied]` field
would be, instead of `ConditionMustHasValidFieldName`; inside a `Segment` every one of them is `FieldDeniedForSegment`.
`MissingContextValue` names no field and no source either. The trace keeps the field, and so does the refusal event
`AuditRefusals` writes (section 22). The Convenience tier still names the field, with `RuleId` and `SourceOrigin`.
The library maps nothing to HTTP. `PolicyException` derives from `LogicException`, so catch it first and keep
deployment faults apart from caller refusals:
```csharp
catch (PolicyException ex) when (ex.ErrorCode is PolicyErrorCode.StoreUnavailable
or PolicyErrorCode.PolicyContextNotPrepared
or PolicyErrorCode.MissingHashSalt
or PolicyErrorCode.MissingTokenVault
or PolicyErrorCode.PolicyRequired
or PolicyErrorCode.TransformRequiresMaterialization)
{
return Results.Problem(ex.Code, statusCode: 500); // misconfiguration or a dependency is down
}
catch (PolicyException ex)
{
return Results.Json(new { error = ex.Code, field = ex.FieldPath }, statusCode: 403); // the policy refuses
}
catch (LogicException ex)
{
return Results.BadRequest(new { error = ex.Message }); // malformed request: an error string of section 8
}
```
---
## 31. Policy recipes and inference channels
### Recipes
- **Tenant boundary:**
- `[DwForceWhere(Operator.Equal, ContextValue = "TenantId")]` on the column.
- `.WithValue("TenantId", tenantId)` on the context.
- `[DwEntity(RequirePolicy = true)]` on the class.
- Optionally `[DwNoWhere]` on the column.
Every query becomes `(caller's group) AND TenantId = x`; a missing value throws `MissingContextValue`.
- **One tenant or none** (3.1.0), such as a system role no institution owns:
`[DwForceWhere(Operator.Equal, ContextValue = "TenantId", AllowNull = true)]` on `int? InstitutionId`.
Every query becomes `(caller's group) AND (InstitutionId = x OR InstitutionId IS NULL)`; a missing value
still throws `MissingContextValue`.
- **Soft delete:** `[DwForceWhere(Operator.Equal, Value = "false")]` on `IsDeleted`, or
`[DwForceWhere(Operator.IsNull)]` on `DeletedAt`.
- **Confirm an identifier, never search or read it** (support agent):
`[DwOperators(Allow = new[] { Operator.Equal, Operator.In })]` +
`[DwMask(MaskStrategy.Partial, KeepEnd = 4)]` + `[DwNoOrder]`.
- The output is `************4242`, and an exact-value filter still matches.
- Without `[DwOperators]`, `StartsWith` plus `TotalCount` sweeps the values.
- Without `[DwMask]`, the value is returned.
- Without `[DwNoOrder]`, sorting ranks the real values.
- **Bands, never values** (analyst):
`[DwGeneralize(GeneralizeMode.Round, Step = 5000, AllowAggregate = true, MinGroupSize = 5)]` +
`[DwNoOrder]`. Values come back rounded, and aggregates only for groups of 5 or more.
- **Join separately run exports on a hidden, high-entropy ID:** `[DwMask(MaskStrategy.Hash)]` +
`[DwNoOrder]`, with the same `HashSalt` (16+ characters) in every pipeline. No shared store is needed.
Whoever holds the salt can recompute every digest.
- **Low-entropy regulated ID with a right to erasure:** `[DwMask(MaskStrategy.Tokenize)]` + `[DwNoOrder]`
+ a durable vault. To erase, delete the vault entry at `DwToken.KeyFor(scope, value)`, where the scope
is `TokenScope` or the field path.
- **One subject across entities:** give every tokenized member that must match the same explicit
`TokenScope`, e.g. `"patient-id"`.
- **Different visibility per role:**
- `[DwDenied]` (sealed) for what no rule may ever grant.
- `[DwMask(..., Overridable = true)]` lets a role rule decide Select and replace the mask stage with
another stage, but never remove it.
- To show one role the raw value, use `[DwMutate(typeof(T))]` and check
`context.Policy.Identities(DwSubjectKind.Role)`.
- **Caller's own names and a filter UI:** `[DwAlias("customer_name")]` +
`[DwDescribe(Label = "Customer", Group = "Identity", Order = 1)]` +
`[DwAllowedValues("Active", "Suspended", "Closed")]`.
- **Refuse unscoped scans, allow scoped ones:** `[DwRequireWhere]` on `Department`.
- **Expensive, sensitive field:** `[DwCost(10)]` + `[DwAudit(PolicyFeature.Select | PolicyFeature.Where)]`.
```
This needs or else
any transform [DwNoOrder] sorting ranks real values; paging reads them
AllowAggregate = true a floor above 1 a group of one returns the exact value
masked but filterable [DwOperators] TotalCount counts matches without selecting
MaskStrategy.Hash HashSalt of 16+ characters MissingHashSalt
MaskStrategy.Tokenize TokenVault MissingTokenVault
tokens that must match one explicit TokenScope the scope follows each field path
any policy [DwEntity(RequirePolicy = true)] a DynamicWhere call without ApplyPolicy reads all
```
### Inference channels
There are eight. The first six let a caller learn a value, or what the policy hides, without reading it;
**Forgotten guard** and **Empty store** are bypasses.
- **Set operations:** `EXCEPT` or `INTERSECT` rebuild a field that is denied for Select but allowed for
Where, from set membership.
- Strict refuses any Segment condition on a select-denied field with `FieldDeniedForSegment`.
- `[DwDeny(PolicyFeature.Segment)]` refuses the field in any Segment, in both tiers.
- **Small-group aggregates:** `MAX`, `MIN` and `SUM` run on real values, so a group of one returns the
exact value. Transformed fields are not aggregatable without `AllowAggregate`, and the group floor
(default 5) removes small groups.
- **TotalCount:** a filter on a protected field counts matches without selecting it. This is inherent to
allowing Where; restrict the field with `[DwOperators(Allow = new[] { Operator.Equal, Operator.In })]`.
- **Sort plus paging:** ordering by a masked field ranks real values, and range filters converge on them.
Use `[DwNoOrder]`; `ValidateModel` warns about every transformed field that can still be sorted.
- **getQueryString:** the SQL names denied columns and the injected predicates. Strict throws
`QueryStringDenied`; Convenience returns the SQL.
- The trace on a result names the same things: `result.Policy` lists the fields a policy dropped, the
attribute or rule that sealed each one, and every injected predicate, and an API that serializes the
result sends it on. Strict leaves it off unless `IncludeTraceInResult = true`; Convenience returns it (3.1.0).
- **Which fields exist:** Convenience answers a name that matches nothing with `ConditionMustHasValidFieldName`
and a denied field with a refusal that names it and its source, so a caller can map the hidden columns one
guess at a time. Strict answers both alike, with the clause's `FieldDeniedFor*` code, `FieldPath` `"*"` and
no source (3.1.0), and closes the side doors too: inside a `Segment` every field refusal is
`FieldDeniedForSegment`, a name padded with dots is normalized as a real path is, `MaxQueryCost` is checked
after the gates so a `[DwCost]` weight cannot set a hidden field apart, and `MissingContextValue` names
neither the scope's column nor its context key (section 17).
- **Forgotten guard:** a code path that never calls `ApplyPolicy` reads everything. Use
`[DwEntity(RequirePolicy = true)]`, which covers only DynamicWhere.ex extension methods.
- **Empty store:** a store with no rules still leaves every attribute enforcing.
- **Not closed by anything:** under `Hash` or `Tokenize`, a caller who can write a chosen value and read it
back learns its stand-in. Use `Fixed`, `Null` or a denial when the column need not group or join.
Posture:
- Use `Strict` unless callers need `getQueryString`.
- Keep the default floor.
- Prefer `Tokenize` with a durable vault over `Hash`.
- Run `DwPolicy.ValidateModel` at startup and treat its warnings as a checklist.
- Put `[DwEntity(RequirePolicy = true)]` on sensitive types.
- Restrict operators rather than allowing free filtering on protected fields.
---
## 32. Traps — policies
Read before generating policy attributes.
1. **A transform without `[DwNoOrder]` leaks through sorting.** ORDER BY runs on the stored value, so
paging ranks the real values. `ValidateModel` only warns.
2. **`AllowAggregate = true` needs a floor above 1.** Aggregation runs in SQL before any transform. The
default `Caps.MinGroupSize` of 5 covers it, but `Caps.MinGroupSize = 1` with no per-field `MinGroupSize`
lets `MAX` over one row return that row's value.
3. **What protects a masked but filterable column is `[DwOperators]`, not the mask.** Allow only `Equal`
and `In`, so a caller can confirm a known value but cannot sweep for one.
4. **A forced predicate wraps the caller's group; it never merges into it.** The result is
`(A OR B) AND TenantId = 5`. Do not hand-build the equivalent.
5. **`[DwDenied]` on a navigation denies only that path; `Contact.Email` stays open.** Either decorate the
members of the navigated type, which applies on every path reaching them (up to 4 segments), or deny
each child path. A `"*"` rule denies every field of the entity, and `"Contact.*"` is not a wildcard.
6. **The default token scope is the field path relative to the queried entity.** Equal member paths on
two entities share tokens, while the same member reached through a navigation gets different tokens.
Set `TokenScope` wherever tokens must or must not match.
7. **Neither `Hash` nor `Tokenize` hides equality.** A caller who writes a value and reads it back learns
its stand-in. `Hash` emits 64 hex characters and `Tokenize` 32, so switching strategies changes the
column width.
8. **Prepare the context once per request.** Since 3.1.0 `ApplyPolicy(context)` raises
`PolicyContextNotPrepared` for a context that skipped `PrepareAsync`, with or without a store; with
attributes alone `PrepareAsync` reads nothing but records that it ran. With a store, adding a `User`
subject after preparation also raises it.
9. **Transforms rewrite the returned instances**, which were loaded `AsNoTracking`. Never attach and save
them. `AsUnguardedQueryable()` output is never transformed.
10. **The group floor removes rows; it never refuses.** It applies to every guarded summary by default
(5). `TotalCount` excludes removed groups, and a result with only small groups is empty.
11. **Policy attributes on public fields are ignored.** `AttributeUsage` allows `Field`, but only
properties are read.
12. **`RequirePolicy` guards only DynamicWhere.ex extension methods.** Plain LINQ or EF on the `DbSet` is
not intercepted.
13. **No runtime rule can unmask a field.** An Allow rule on Select leaves the mask running. A sealed
transform also adds a sealed Mask on Select, so no rule can deny Select either; mark the transform
`Overridable` if a rule must be able to.
14. **`Overridable` does nothing on `[DwOperators]` or `[DwForceWhere]`.** A rule cannot widen a sealed
`[DwAudit]` either.
15. **Aliases rename output columns only in dynamic filter results and summaries.** Typed results keep
member names.
16. **`[DwAudit]` records only fields the query uses by name: those the request names, and a `DefaultOrder`
field it orders by (3.1.0).** A request without `Selects` returns audited columns with no Select event, and a
default field left out for the caller records nothing.
17. **`AmbiguousGroupKey` compares only the transformed keys.** A summary mixing untransformed and
transformed keys can be refused even though its rows differ.
18. **`[DwFormat]` ignores its format for a string value.** `[DwGeneralize(GeneralizeMode.Round, ...)]`
plus `[DwFormat("C0")]` on a string member returns the plain rounded number.
19. **The composable `Group` and `Summary` return a query you materialize yourself.** The floor and the forced
predicates are applied, but nothing transforms or renames those rows, and a type with any transform for
this caller refuses both with `TransformRequiresMaterialization`.
20. **`ValidateModel` does not catch a malformed `[DwAlias]` name.** A blank, dotted or `"*"` alias throws
`ArgumentException` on every guarded query of the type, and of any type that navigates to it. Since 3.1.0
it does report a malformed `[DwForceWhere]`.
21. **Raising `Caps.MaxNavigationDepth` above 4 opens paths no attribute covers.**
`AttributePolicyProvider.MaxDepth` is a constant 4, so a member 5 or more segments deep is allowed
and untransformed.
22. **A guarded query over an in-memory source masks your objects.** `list.ApplyPolicy(ctx).ToList(filter)` without
`Selects` returns the list's own instances and transforms them in place, so the list stays masked afterwards.
EF Core queries run `AsNoTracking` and are unaffected. Query a copy, or send `Selects`.
23. **Dry run returns what the policy would withhold.** Denied fields come back, forced predicates are not applied
(rows outside a tenant scope appear), caps and cost do not refuse. Only transforms still run. Never enable it
for callers who must not see everything.
24. **A store provider renews its snapshot from the poll, not only from a reload.** A poll that reads back the
version being served stamps the load time and clears the degraded flag. With `autoRefresh: false` nothing
renews it: call `RefreshAsync` more often than `MaxSnapshotAge`, or every guarded query starts throwing
`StoreUnavailable`. A context pinned before a renewal is refused either way — prepare one per request.
25. **`new PolicyResolver(...)` does not include `AttributePolicyProvider`.** With the four-argument `ApplyPolicy`,
`PolicySimulator` or `PolicySchemaBuilder`, attributes are ignored unless the list contains
`new AttributePolicyProvider()`. `DwPolicy.Resolver` always contains it.
26. **`ApplyPolicy` works without `DwPolicy.Configure`** — on a frozen default: Convenience tier, floor 5, no salt,
no vault, no store. A startup path that forgets `Configure` silently runs the weaker tier.
27. **`DwPolicy.ValidateModel` throws when any error exists** (`InvalidOperationException` listing all of them), so
code that checks `report.Errors` afterwards never runs. Use `PolicyModelValidator.Inspect` to get the report
without throwing; log its `Warnings`.
28. **An existing `catch (LogicException)` also catches policy refusals and deployment faults** (`StoreUnavailable`,
`MissingHashSalt`, `MissingTokenVault`, `PolicyContextNotPrepared`). Catch `PolicyException` first (section 30).
29. **Guarded dynamic and summary rows become `ExpandoObject`** when an alias renames a column or the group floor
applied (every guarded `ToList(Summary)` with the default floor). System.Text.Json writes their keys as they
are (`"Name"`, `"dept"`), not in the camelCase of the envelope.
30. **`AddDwPolicies` builds its own options instance.** A `StorePolicyProvider` reads `StoreFailure`,
`MaxSnapshotAge` and `RefreshInterval` from the options given to `CreateAsync`; with a store, bind and configure
by hand (section 12).
31. **POST /rules cannot write transforms, operator lists, forced predicates or facts.** Its body has no field for
them. Write such rules with `IDwPolicyWritableStore.UpsertAsync(new PolicyRule(...))`.
32. **A stored rule can throw on every query.** No store and not POST /rules validates a rule's alias or its forced
predicate's field; a bad one is saved and then throws `ArgumentException` on the query path for every caller it
applies to. Build rules in a test with `new PolicyRule(...)` and `ToFragment()` first.
33. **Under the Strict tier `result.Policy` is null.** Since 3.1.0 a guarded result carries the trace only when
`DwPolicyOptions.IncludeTraceInResult` allows it, and null follows the tier: off under Strict. Read
`PolicyQueryable<T>.LastTrace` in-process. Setting the option to true sends the dropped fields, the attributes
that sealed them and every injected predicate to whoever reads the result.
34. **Under the Strict tier a misspelt field is a policy refusal, not a validation error.** Since 3.1.0 a path that
matches nothing throws `PolicyException` with its clause's `FieldDeniedFor*` code, exactly as a denied field
does, instead of `LogicException("ConditionMustHasValidFieldName")`; under the section 30 mapping that is a
403, not a 400. Its `FieldPath` is `"*"` and `RuleId` and `SourceOrigin` are null, as on every such refusal,
so never build a message or a log line from them; a `PolicySimulator` trace and the `AuditRefusals` event
name the field.
35. **`AllowNull = true` widens the rows, not the caller, and never satisfies `[DwRequireWhere]`.** A context
without the value is still refused with `MissingContextValue`. The injected
`(field op value OR field IS NULL)` is an `Or`, not a narrowing condition, so a `[DwRequireWhere]` on the
same member still demands the caller's own filter.
36. **With `AuditRefusals` on, a sink receives refusals beside uses.** An event whose `ErrorCode` is not null
records a refused query: its `Effect` is `Deny`, its `FieldPath` can be `"*"` or a name that matches nothing
on the type, and it is written whether or not any field carries `[DwAudit]`. Branch on `ErrorCode` before
counting an event as an access.
37. **`[DwEntity(DefaultOrder = ...)]` orders only guarded queries, and is never a tiebreak.** Unguarded calls
ignore it. A caller who sends any `Orders` gets exactly those, so rows tied on them can still move between
pages, and an `IQueryable` ordered before `ApplyPolicy`, or by a composed `Order` before `Page`, keeps its own
order; a list sorted in memory before `ApplyPolicy` is not seen as ordered and gets the default, so send the
order with the filter. A projected query takes the default only when its outermost `Select` builds T in an
object initializer assigning every default field a column (3.2.0); a computed value, even one EF Core could
translate, leaves it unordered, and the guarded `Select` composed afterwards never takes it.
A default field the caller may not order by is left out without an error. End every
order meant for paging with a unique field, such as the key.
38. **A `[DwEntity]` on a derived type replaces its base type's.** The attribute allows one per type, and .NET
inheritance hands a derived type its own when it declares one, so the base type's `RequirePolicy` and
`DefaultOrder` are gone rather than merged. `[DwEntity(DefaultOrder = "Id")]` on a subclass of a
`RequirePolicy` type lets a DynamicWhere call on the subclass run without `ApplyPolicy`. Repeat every setting on
the derived type.
---
## 33. Reflection cache
The cache is automatic; tuning it is optional. The query engine and the policy layer look up type
members through one static, thread-safe cache: every Filter, Segment and Summary validation and every
expression build. Nothing has to be registered or called. Without `CacheExpose.Configure` the defaults
apply.
- One cache per process, shared by every DbContext, request and thread.
- It starts empty and is never shared between app instances.
- `CacheExpose` is a static class, so there is nothing to inject.
```
Store (CacheMemoryType) Key Value Holds
TypeProperties Type Dictionary<string, PropertyInfo> public instance properties, keys OrdinalIgnoreCase
PropertyPath (Type, string) string raw path in, declared-casing path out; successes only
CollectionElementType Type Type? element type; null when not a recognized collection
```
- It holds only these three stores: no compiled expressions, no LINQ strings, no query results.
- The policy layer has its own static caches: attribute fragments per type, `[DwMutate]` transformer
instances, and compiled getters and setters. `CacheOptions` does not size them, and `CacheExpose`
neither reports nor clears them.
- A store is filled on the first lookup that misses. `WarmupCache` fills stores ahead of traffic.
```
DynamicWhere.ex.Optimization.Cache.Source CacheExpose every other class in this namespace is internal
DynamicWhere.ex.Optimization.Cache.Config CacheOptions
DynamicWhere.ex.Optimization.Cache.Enums CacheEvictionStrategy CacheMemoryType
DynamicWhere.ex.Optimization.Cache.DTOs CacheStatistics CacheConfiguration CacheMemoryUsage
CachePerformanceEvaluation CacheMonitoringSession
DynamicWhere.ex.Optimization.Cache.Input HealthAlertsInput CacheFullCheckInput AccessTrackingInput<TKey> MemoryCalculationInput
DynamicWhere.ex.Optimization.Cache.Output CacheCounts TrackingCounts CacheDatabases
```
### Cache enums — verbatim
```
CacheEvictionStrategy FIFO=0 LRU=1 LFU=2 LRU is the default
CacheMemoryType TypeProperties=0 PropertyPath=1 CollectionElementType=2
```
### CacheOptions
```
MaxCacheSize int 1000 > 0 cap per store, not in total
LeastUsedThreshold int 25 1–50 % of a store removed per eviction pass
MostUsedThreshold int 75 50–99 must equal 100 − LeastUsedThreshold; eviction never reads it
EvictionStrategy CacheEvictionStrategy LRU
EnableLruTracking bool true reported only; auto-validation overwrites it
EnableLfuTracking bool false reported only; auto-validation overwrites it
AutoValidateConfiguration bool true correct inconsistencies instead of throwing
Validate() -> void ArgumentOutOfRangeException / ArgumentException; may modify this instance
Clone() -> CacheOptions
```
`Configure` calls `Validate()`, which checks in this order:
1. A value out of range throws `ArgumentOutOfRangeException`, with or without auto-validation.
2. If the thresholds do not sum to 100: with auto-validation, `MostUsedThreshold` becomes
`100 − LeastUsedThreshold`; without it, `ArgumentException`.
3. Tracking flags: with auto-validation they are set from the strategy (FIFO false/false, LRU true/false,
LFU false/true). Without it, a flag that is true for a strategy that does not use it throws
`ArgumentException`.
- Set `LeastUsedThreshold` on its own. The correction only runs in that direction, and an out-of-range
`MostUsedThreshold` throws even though it would have been overwritten.
- With `AutoValidateConfiguration = false`, FIFO and LFU must also set `EnableLruTracking = false`,
because its default of `true` throws.
- There is no `CacheOptions.Default`; the default is `new CacheOptions()`.
### Presets — static factories on CacheOptions, each returning a new mutable instance
```
MaxCacheSize LeastUsed MostUsed Strategy Intended for
new CacheOptions() 1000 25 75 LRU general default
ForHighMemoryEnvironment() 5000 10 90 LRU high memory, conservative eviction
ForLowMemoryEnvironment() 250 40 60 LFU low memory, aggressive eviction
ForDevelopment() 100 50 50 FIFO development and testing
ForHighFrequencyAccess() 2000 20 80 LFU repeated access to the same items
ForTemporalAccess() 1500 25 75 LRU recent-access patterns
```
Every preset sets `AutoValidateConfiguration = true`. The tracking flags keep their defaults until
`Configure` validates the options.
### Configure
```csharp
using DynamicWhere.ex.Optimization.Cache.Config; // CacheOptions
using DynamicWhere.ex.Optimization.Cache.Enums; // CacheEvictionStrategy, CacheMemoryType
using DynamicWhere.ex.Optimization.Cache.Source; // CacheExpose
CacheExpose.Configure(CacheOptions.ForHighMemoryEnvironment());
// or: the action receives new CacheOptions(), not the active options
CacheExpose.Configure(o => { o.MaxCacheSize = 2000; o.EvictionStrategy = CacheEvictionStrategy.LFU; });
// or: adjust a preset
var options = CacheOptions.ForLowMemoryEnvironment();
options.MaxCacheSize = 500;
CacheExpose.Configure(options);
CacheExpose.WarmupCache<Customer>("Contact.Email", "Orders.Items.Sku"); // after Configure
CacheExpose.WarmupCache(typeof(Customer), "Name");
```
- Callable at any time, from any thread, any number of times. Each call replaces the previous options.
- Validation runs before the swap: if it throws, the active options stay. A null argument throws
`ArgumentNullException`.
- A copy is stored, so later edits to your instance do nothing. `GetCacheConfigOptions()` also returns
a copy.
- Each cache call reads the options once when it starts, so calls already running finish on the old ones.
- Existing entries stay.
- A lowered `MaxCacheSize` trims one eviction pass per later miss.
- `ForceEvictionOnAllCaches()` trims immediately.
- There is no `IConfiguration` binding and no DI registration for the cache; configure it in code.
- Configure and Clear act process-wide, parallel tests included. `ClearAllCaches()` keeps the options;
`Configure(new CacheOptions())` restores the defaults.
### Eviction
- Runs only when a lookup misses in that store and the store already holds more than `MaxCacheSize`
entries. The pass runs before the new entry is added, so a store reaches `MaxCacheSize + 1` entries,
or more when misses happen concurrently.
- Removes `max(1, count × LeastUsedThreshold / 100)` entries (integer division), from that store only.
- FIFO removes the first keys in `ConcurrentDictionary` enumeration order. That type keeps no insertion
order, so FIFO does not remove the oldest entries first.
- LRU removes the oldest last-access ticks first. LFU removes the lowest access counts first, breaking
ties by key hash code. Both consider only entries that have a record of their own kind, and delete the
record with the entry.
- An undefined strategy value, or an exception during eviction, falls back to FIFO removing 50% of the store.
### Access tracking
- These calls record an access before they look anything up: `GetTypeProperties`, `FindProperty`,
`GetCollectionElementType` and `IsCollectionType`. `ValidatePropertyPath` records one only once the path has
validated, so a path that fails validation records nothing (3.1.0). The query engine's own calls count too.
- LRU writes `DateTime.UtcNow.Ticks`, LFU adds 1, and FIFO records nothing. Only `EvictionStrategy`
decides this; the `Enable*Tracking` flags have no runtime effect.
- A record is written even when the key never enters the store, such as an internal field-type lookup.
Eviction only deletes records of entries it removes, so `TrackingCounts` can exceed `CacheCounts`.
- Fixed in 3.1.0: `ValidatePropertyPath` recorded the access before validating, so under LRU (the default) or
LFU every distinct invalid field name a caller sent stayed recorded for the life of the process, or until
`ClearCache` / `ClearAllCaches`. A caller sending unique invented names grew the process without limit, faster
under the Strict tier, which resolves every unknown name of a request. A failed path now leaves no record.
- Changing strategy leaves the old records in place. An entry with no record for the current strategy is
never evicted. Example: an entry cached under FIFO and not read since the switch to LRU. Warm up after
`Configure` for this reason.
- Statistics, counts, reports and alerts do not record accesses.
### CacheExpose — every public member
```
Configuration
Configure(CacheOptions options) -> void
Configure(Action<CacheOptions> configureOptions) -> void
GetCacheConfigOptions() -> CacheOptions a copy
Reflection — reads through the cache, fills it, records an access
GetTypeProperties(Type type) -> Dictionary<string, PropertyInfo>
FindProperty(Type type, string propertyName) -> PropertyInfo?
IsCollectionType(Type type) -> bool
GetCollectionElementType(Type type) -> Type?
ValidatePropertyPath(Type rootType, string propertyPath) -> string
WarmupCache<T>(params string[] commonPropertyPaths) -> void
WarmupCache(Type type, params string[] commonPropertyPaths) -> void
Statistics and reports
GetCacheStatistics() -> CacheStatistics
GetCacheConfiguration() -> CacheConfiguration
GetMemoryUsage() -> CacheMemoryUsage
EvaluatePerformance() -> CachePerformanceEvaluation
CreateMonitoringSession() -> CacheMonitoringSession
GenerateHealthAlerts(HealthAlertsInput input) -> List<string>
GenerateMonitoringReport() -> Dictionary<string, object>
GeneratePerformanceReport() -> string
GenerateCompactStatusReport() -> string
GenerateCacheAnalysisReport() -> string
GetQuickHealthSummary() -> string
Management
ClearAllCaches() -> void
ClearCache(CacheMemoryType cacheType) -> void
ForceEvictionOnAllCaches() -> void
GetCacheCounts() -> CacheCounts
GetTrackingCounts() -> TrackingCounts
IsCacheFull(CacheMemoryType cacheType) -> bool
IsCacheFull(CacheFullCheckInput input) -> bool
IsEvictionNeeded(CacheMemoryType cacheType) -> bool
CalculateEvictionCount(int currentCacheSize) -> int
GetEvictionStrategyDescription() -> string
Utilities
FormatBytes(long bytes) -> string
GetMemorySizeConstants() -> Dictionary<string, long>
CalculateStringSize(string str) -> long
```
Reflection members:
- `GetTypeProperties` returns the cached dictionary itself, including properties inherited from base
classes. Never modify it. When two names are equal ignoring case, the property reflection lists last wins.
- `FindProperty` looks up one name, case-insensitively. A dotted name returns null.
- `GetCollectionElementType` recognizes arrays, plus generic types whose definition is exactly `List<>`,
`ICollection<>`, `IEnumerable<>`, `IList<>`, `HashSet<>` or `ISet<>`.
- Everything else returns null, including `IReadOnlyCollection<>`, `IReadOnlyList<>`, `Collection<>`,
a class deriving from `List<T>`, and `string`.
- `IsCollectionType(t)` is `GetCollectionElementType(t) != null`.
- `ValidatePropertyPath`:
- throws `LogicException` with message `"FieldPath[<path>]StartsWithReservedName"` when the first segment is one
of the parser's own words (3.1.0, section 5). Checked before anything is looked up, so nothing is cached and
no access is recorded;
- splits on `.`, trims each segment and drops empty ones;
- matches segments case-insensitively, stepping into the element type of a recognized collection;
- returns the declared names joined by `.`, e.g. `" contact . EMAIL"` → `"Contact.Email"`;
- throws `LogicException` with message `"ConditionMustHasValidFieldName"` when a segment is missing.
Failures are not cached, and each raw spelling is stored as its own entry.
- `WarmupCache` caches the type's properties, then validates each path.
- A failing path is skipped silently, and a null array is allowed.
- The first validation of a path also caches every type it walks, and the collection check of each
segment's property type (null for non-collections).
Management members:
- `ClearAllCaches()` empties all three stores and all six tracking dictionaries. `ClearCache` empties one
store and its two tracking dictionaries. Queries then refill the stores as they run.
- `IsCacheFull(CacheMemoryType)` and `IsEvictionNeeded` run the same test: count strictly greater than the
active `MaxCacheSize`, not equal to it. `IsCacheFull(CacheFullCheckInput)` tests against `input.MaxSize`
and returns false when that is ≤ 0.
- `CalculateEvictionCount(n)` is `max(1, n × LeastUsedThreshold / 100)` under the active options.
- `ForceEvictionOnAllCaches()` runs one eviction pass on each store, whatever its size.
- `FormatBytes` returns `"n B"` under 1024, then `"x.x KB"`, `"x.xx MB"`, `"x.xxx GB"`.
- `CalculateStringSize(s)` is `24 + 2 × s.Length`, or 0 for null or empty.
- `GetCacheCounts` and `GetTrackingCounts` only read counts. Every call that returns memory figures walks
every entry of every store: `GetCacheStatistics`, `GetMemoryUsage`, and every report, alert and evaluation.
### Memory figures are estimates
They are computed from fixed 64-bit constants, not measured from the GC. `GetMemorySizeConstants()`
returns these constants:
```
ObjectReference 8 StringOverhead 24 DictionaryOverhead 72 ConcurrentDictionaryOverhead 256 DictionaryEntryOverhead 32
ConcurrentDictionaryEntryOverhead 48 TupleOverhead 24 LongValue 8 PropertyInfoSize 200 NullableByte 1
each store and each tracking dictionary: 0 when empty, otherwise 256 plus
TypeProperties 128 per type + (264 + 2 × name length) per property
PropertyPath 128 + 2 × (input length + output length) per entry
CollectionElementType 65 per entry
tracking dictionary 64 per Type-keyed record; 112 + 2 × path length per path-keyed record
```
### Result types
`*` marks a computed, read-only member. Every type except `CacheMonitoringSession` is a mutable snapshot
that does not update.
```
CacheCounts
int TypePropertiesCount PropertyPathCount CollectionTypeCount TotalCachedEntries*
static FromValues(int typePropertiesCount, int propertyPathCount, int collectionTypeCount) GetSummary() -> string
TrackingCounts
int TypeAccessRecords PathAccessRecords CollectionAccessRecords LRU
int TypeFrequencyRecords PathFrequencyRecords CollectionFrequencyRecords LFU
int TotalLruRecords* TotalLfuRecords* TotalTrackingRecords*
static FromValues(int typeAccessRecords, int pathAccessRecords, int collectionAccessRecords,
int typeFrequencyRecords, int pathFrequencyRecords, int collectionFrequencyRecords) GetSummary() -> string
CacheStatistics
int TypePropertiesCount PropertyPathCount CollectionTypeCount TotalCachedEntries*
int TypeAccessRecords PathAccessRecords CollectionAccessRecords
int TypeFrequencyRecords PathFrequencyRecords CollectionFrequencyRecords TotalTrackingRecords*
long TypePropertiesMemoryBytes PropertyPathMemoryBytes CollectionTypeMemoryBytes
LruTrackingMemoryBytes LfuTrackingMemoryBytes TotalMemoryBytes*
double TypePropertiesMemoryMB* PropertyPathMemoryMB* CollectionTypeMemoryMB*
LruTrackingMemoryMB* LfuTrackingMemoryMB* TotalMemoryMB* 3 decimals
CalculateUtilizationPercentage(int maxCacheSize) -> double mean of the three stores' count ÷ max × 100; 0 when max ≤ 0
CalculateMemoryEfficiency() -> double entries per MB; 0 when TotalMemoryMB is 0
CalculateAverageEntrySize() -> double bytes per entry; 0 when empty
GetMemoryDistribution() -> Dictionary<string, double> percent, 1 decimal; empty when total is 0
GetSummary() -> string
static FromValues(the 14 settable properties above, in that order, as camelCase parameters)
CacheMemoryUsage
long TypePropertiesMemory PropertyPathMemory CollectionTypeMemory LruTrackingMemory LfuTrackingMemory
long TotalMemory* CacheOnlyMemory* TrackingOnlyMemory*
double TotalMemoryMB* CacheOnlyMemoryMB* TrackingOnlyMemoryMB* 3 decimals
GetMemoryDistribution() -> Dictionary<string, double> percent, 2 decimals; empty when total is 0
CalculateTrackingOverheadPercentage() -> double
CalculateCacheEfficiencyRatio() -> double cache ÷ tracking; +∞ when there is no tracking memory
GetLargestMemoryConsumer() -> (string ComponentName, long MemoryBytes)
EvaluateMemoryHealthStatus(double warningThresholdMB = 50.0, double criticalThresholdMB = 100.0) -> string
GetOptimizationRecommendations() -> List<string> never empty
GetDetailedSummary() -> string GetCompactSummary() -> string
static FromValues(long typePropertiesMemory, long propertyPathMemory, long collectionTypeMemory,
long lruTrackingMemory, long lfuTrackingMemory) static Empty()
CacheConfiguration
int MaxCacheSize LeastUsedThreshold MostUsedThreshold
string EvictionStrategy "FIFO" | "LRU" | "LFU"
bool EnableLruTracking EnableLfuTracking AutoValidateConfiguration IsTrackingEnabled*
string EvictionStrategyDescription* MemoryOverhead* "Minimal" | "Low (timestamp tracking)" | "Low (frequency tracking)"
ValidateConfiguration() -> List<string> issues; empty when consistent; never throws
GetSummary() -> string
static FromValues(int maxCacheSize, int leastUsedThreshold, int mostUsedThreshold, string evictionStrategy,
bool enableLruTracking = false, bool enableLfuTracking = false, bool autoValidateConfiguration = true)
CachePerformanceEvaluation
CacheOptions Configuration CacheStatistics Statistics CacheMemoryUsage MemoryUsage
double PerformanceScore List<string> Recommendations List<string> HealthAlerts DateTime Timestamp (UTC)
GetSummary() -> string
CacheMonitoringSession not thread-safe
new CacheMonitoringSession() starts the clock
RecordSnapshot() -> void appends CacheExpose.EvaluatePerformance()
GetPerformanceTrend() -> string first snapshot against last; needs two or more
GetHistory() -> List<CachePerformanceEvaluation> a copy
```
- Distribution dictionaries use the keys `TypeProperties`, `PropertyPaths`, `CollectionTypes`,
`LruTracking` and `LfuTracking`.
- `EvaluateMemoryHealthStatus` returns an emoji icon followed by one of:
- `CRITICAL: {MB:F2} MB (>{critical} MB)` at or above the critical threshold;
- `WARNING: …` at or above the warning threshold;
- `HEALTHY: {MB:F2} MB (<{warning} MB)` otherwise.
- `PerformanceScore` ranges 0–100 and is the mean of four scores:
- `min(100, utilization%)`
- `min(100, entries per MB ÷ 10)`
- `100 − tracking overhead%`
- health at 50/100 MB: 100 healthy, 70 warning, 30 critical
- Each of these adds a recommendation:
- tracking overhead above 30%
- TypeProperties above 60% of memory
- PropertyPaths above 40% of memory
- total above 100 MB, or above 50 MB
- cache ÷ tracking below 2
- when none applies, a single "optimal" line
- `GetPerformanceTrend` reports the first condition that holds:
- score change > +5: improving
- score change < −5: declining
- memory growth > 10 MB: memory increasing
- entries growth > 1000: cache growing
- otherwise: stable
### Health alerts and monitoring data
```
HealthAlertsInput
CacheOptions Config (required) double WarningThresholdMB = 50.0 double CriticalThresholdMB = 100.0
static WithDefaults(CacheOptions config) -> HealthAlertsInput
static Create(CacheOptions config, double warningThresholdMB, double criticalThresholdMB) -> HealthAlertsInput
IsValid() -> bool Config not null, both thresholds > 0, critical > warning
GetSummary() -> string
CacheFullCheckInput
CacheMemoryType CacheType int MaxSize
static Create(CacheMemoryType cacheType, int maxSize) -> CacheFullCheckInput
static FromConfig(CacheMemoryType cacheType, CacheOptions config) -> CacheFullCheckInput MaxSize = config.MaxCacheSize
IsValid() -> bool MaxSize > 0
```
`GenerateHealthAlerts` returns one string per rule that fires. An empty list means no rule fired.
```
total memory ≥ CriticalThresholdMB, else ≥ WarningThresholdMB CRITICAL / WARNING
mean store utilization ≥ 90% of Config.MaxCacheSize WARNING
tracking overhead ≥ 40% WARNING
fewer than 50 entries per MB WARNING always fires on an empty cache
a store's count ≥ 95% of Config.MaxCacheSize WARNING "<store> cache is near capacity", per store
input fails IsValid() one "ERROR: Invalid health alerts input parameters" item, no throw
```
- Build the input from `CacheExpose.GetCacheConfigOptions()`. A bare `new HealthAlertsInput()` has a null
`Config` and returns only the error item.
- Icons in the output are broken:
- alert strings and `GetQuickHealthSummary()` start with a literal `?` or `??`;
- the performance and analysis reports use U+FFFD as bullets and `?` as chart bars.
Match on the words `CRITICAL`, `WARNING` and `ERROR`, never on the icons.
- `GetQuickHealthSummary()` returns `"<icon> <entries> entries, <FormatBytes(total bytes)>"`.
- `GenerateCompactStatusReport()` returns
`"Cache Status: n entries | Utilization: x% | Memory: yMB | Strategy: S | Health: <EvaluateMemoryHealthStatus()>"`.
- `GeneratePerformanceReport()` contains:
- configuration
- entries, utilization, memory and efficiency
- the detailed memory summary
- recommendations
- `GenerateCacheAnalysisReport()` contains:
- each store's count against `MaxCacheSize`
- tracking record counts
- eviction size, and whether each store needs eviction
- memory distribution
`GenerateMonitoringReport()` keys:
```
timestamp DateTime (UTC) cache_strategy string health_status string
total_entries int total_memory_bytes long
total_memory_mb memory_efficiency utilization_percentage tracking_overhead_percentage cache_efficiency_ratio double
type_properties_count property_path_count collection_type_count int
type_access_records path_access_records collection_access_records int
type_frequency_records path_frequency_records collection_frequency_records int
```
### Public types that reach nothing
No `CacheExpose` member accepts or returns these types. Building one does not touch the live cache.
```
AccessTrackingInput<TKey> where TKey : notnull
TKey Key CacheOptions Config ConcurrentDictionary<TKey, long> AccessTimes ConcurrentDictionary<TKey, long> AccessCounts
static Create(TKey key, CacheOptions config, ConcurrentDictionary<TKey, long> accessTimes,
ConcurrentDictionary<TKey, long> accessCounts) IsValid() -> bool
CacheDatabases
ConcurrentDictionary<Type, Dictionary<string, PropertyInfo>> TypePropertiesCache
ConcurrentDictionary<(Type, string), string> PropertyPathCache
ConcurrentDictionary<Type, Type?> CollectionElementTypeCache
ConcurrentDictionary<Type, long> TypePropertiesAccessTime
ConcurrentDictionary<(Type, string), long> PropertyPathAccessTime
ConcurrentDictionary<Type, long> CollectionElementTypeAccessTime
ConcurrentDictionary<Type, long> TypePropertiesAccessCount
ConcurrentDictionary<(Type, string), long> PropertyPathAccessCount
ConcurrentDictionary<Type, long> CollectionElementTypeAccessCount
static FromDictionaries(the nine above, in that order, camelCase) GetCacheCounts() GetTrackingCounts()
AreAllDatabasesInitialized() -> bool
MemoryCalculationInput
the same nine properties
static Create(the nine, in that order, camelCase) static FromDatabases(CacheDatabases databases) IsValid() -> bool
GetCacheCounts() GetTrackingCounts() GetMeasurementSummary() -> string
```
---
## 34. Version history, breaking changes and limits
### History
```
3.2.0 A projected row keeps its members, denials the gate could not see are enforced, a default order that
reaches projected rows, and a CancellationToken on every async terminal. The security fixes refuse or
withhold what 3.1.0 returned; the bullets marked "Behaviour change" also change what a correct query
returns; and one call form stops compiling (the token bullet).
- Security fix and behaviour change. With no Selects, a field denied for Select only beneath a member, none at
the top of T, synthesized no projection, so the whole row came back with the denied value in it: in a list or
nested object of a row projected before ApplyPolicy, in a row in memory, and in an entity's included,
automatically included, lazily loaded or owned member. Such a denial now synthesizes the projection when its
value can reach the result, in both tiers, typed and dynamic, for a Filter and a Segment. On an entity that
means beneath a column, an owned or complex member, or a navigation the query loads; a denial beneath a
navigation nothing loads never leaves the database, and the entity is read as in 3.1.0.
- Security fix. Where the query hides what loads, every navigation now counts as loaded: an include named from
the root and re-rooted by Select(o => o.Customer), SelectMany or Join, which EF Core still applies, and a
projection behind another Select (an identity Select, a member of an anonymous row, a conditional). So does
a lazy loader the constructor takes, delegate or ILazyLoader, kept in a field or a property of any name, and
an initializer after a constructor with arguments counts every member as assigned. So do an injected
DbContext and EF Core 7's asynchronous loader delegate. So does a reshaped chain whose lambda hands its rows
an object an application's method returns from the row, or one it captured: another query with its own
include or projection, or an object in memory. Each returned the denied value. A reshaped chain with none of
these is still read from the model, so it is not projected for a denial beneath a navigation it does not
load. A specification, a repository's query, FromSql and a context's Set through an interface are evaluated
as EF Core evaluates them, a context's query function is a query root, and an anonymous object carrying
range variables or a composite key, or a value that only feeds a predicate or a key, builds nothing.
- Security fix. A guarded query through a provider that wraps EF Core's, LinqKit's AsExpandable or
DelegateDecompiler's Decompile, ran tracking: EF Core's AsNoTracking hands such a query back unchanged. The
rows' navigations were then filled from entities the context already tracked, the denied ones included,
and a masked value became a pending change the next SaveChanges would write. AsNoTracking now goes into the
query itself.
- Security fix and behaviour change. A member declared as a base type or an interface holds its subtypes,
whose denied fields the declared type never names. They are read now: the types the EF Core model derives,
for an entity, and every loaded subtype for a projected or in-memory row, an open generic one and an
application's subclass of a framework class included. A query over the root of a hierarchy whose derived
type declares a denied field, an included or named base-typed navigation, a base-typed member of a row, and
a projection constructing a subtype of T, all returned it. Such rows are projected to T and such members
narrowed to the declared type, which drops the subtype's fields, allowed ones too. Over an abstract T the
typed terminals then fail with SelectTypeMustHaveParameterlessConstructor, as for any T they cannot build;
the dynamic terminals return the root's allowed members. A rule on a subtype's field through a base-typed
member was dropped as naming nothing; it is enforced.
- Security fix. A deny-family attribute on an override, on a public member a subtype hides with new, on the
implementation of an interface member (through a variant instantiation too), or on the interface member a
class implements, was read only from its own declaration, so the base type's or the interface's path
filtered, sorted, grouped and returned the value. It applies to the path now.
- Security fix. Under a "*" deny with exact allows, a path the walk never asked about (past four segments,
around a cycle, with no setter, on a subtype) resolved as allowed, so a member holding one was returned
whole, named or not. Each such path is asked of the policy now; one it does not name is denied, and a
member holding one is refused, narrowed or projected as one with a denial beneath it is.
- Security fix. A field denied at the top of T whose type is not a simple value (a blob, a list, an owned
object or a JSON column, say) synthesized no projection either, so with nothing else denied it came back.
- Security fix. The attribute walker read any namespace starting with "System" as the framework's, so an
application's SystemsCorp.Payroll got no fragment beneath its types, and a [DwDenied] field there was
returned, filterable and sortable. Only System and the namespaces beneath it are the framework's now.
- Security fix. Under Convenience, Selects naming a navigation whose element key (Id) is denied narrowed the
key away, and the core's typed projection added it back. Such a narrowing is refused with
FieldDeniedForSelect in both tiers, as naming a sibling of the key already was. A navigation named through
another, Main.Lead, now gates Main's key, which the builder adds; it did not.
- Security fix. Selects naming a member typed as a collection the core does not unwrap (IReadOnlyList<T>,
IReadOnlyCollection<T>, Collection<T> or an application's own) returned every field beneath it, denied ones
included, in both tiers: the projection gate read collections through a narrower list than the attribute
walker. It reads them as the walker does, and a narrowing the core cannot project is refused.
- Security fix. Selects naming a member that carries a field denied for Select no path names returned it:
deeper than four segments, inside a framework generic such as Dictionary<string, T>, or, on a named entity
navigation, in its owned chain or a converted column. What a member carries is read from the source, from
the EF Core model for an entity. Strict refuses it; Convenience narrows it where the core can, and refuses it
where it cannot. A denied property with no setter, and a rule on a path reached through a cycle, are found
beneath a named member too.
- Behaviour change. The synthesized projection keeps what the source carries. A row a projection builds keeps
its assigned nested objects and lists; an entity keeps its columns, converted and JSON ones included, and its
owned and complex members, and every member holding a collection of simple values (byte[], List<string>). In
3.1.0 all of these came back null or empty whenever a field was denied. A member is kept whole when nothing
it can hold is denied, narrowed around a denial where the core's narrowing translates, and otherwise left out
whole with a "left out whole" Dropped decision. An entity's navigations and the objects of a row in memory
are left out, as before, and each one the unguarded call would have returned is recorded as Dropped; a
value EF Core does not map is left out too. A member that can hold an object of any type, a geometry or a
JSON bag say, asks for no projection on its own, and an entity keeps it whole, unless a value converter
hands back its value, directly or inside a complex property: a converter is the application's code, so
such a column is left out. BitArray and the framework's string collections hold values. An application's
own collection class, generic or not, has its own members read; a collection of values stays a value unless
one of them is denied. Two members sharing a name, one hidden with new under another type or spelled in
another case, are left out when either holds a denial, since the core reads one and a row carries both.
Rows in memory are projected when a member a base type declares, and the row type hides with new, is
denied. A projected member is read as the type its initializer constructs, and an
initializer after a constructor with arguments narrows its own bindings. The trace records a member left
out only when a projection is built, or, in a dry run, would be.
- Behaviour change. [DwEntity(DefaultOrder)] applies to a projected source whose outermost Select builds T
in an object initializer assigning every field the default names a column: a mapped member, read directly,
through reference navigations or through EF.Property. A computed value or any other projection still leaves
the query in its own order. A Select, or a Filter with Selects, composed on the guarded handle keeps the rest
of the chain unordered. A composed Filter that sent orders gets no default later in the chain, as a composed
Order already did not.
- Every async terminal has overloads taking a CancellationToken, guarded and unguarded: ToListAsync and
ToListAsyncDynamic with a Filter, ToListAsync with a Summary, and ToListAsync with a Segment. The 3.1
signatures are unchanged, so code compiled against 3.1 still binds. The token reaches the count and the
read. ToListAsync(filter, default) no longer compiles, since default fits both bool and CancellationToken,
and a reflection lookup of one of these methods by name alone finds more overloads than it did.
- Behaviour change. The async Summary counts through EF Core's CountAsync, where it counted synchronously, and
it and ToListAsyncDynamic read through EF Core's ToListAsync instead of Dynamic LINQ's ToDynamicListAsync,
so on an EF Core query a canceled token reaches the database. A provider that is not EF Core's keeps Dynamic
LINQ's read, on the calling thread.
3.1.0 Dates rebuilt, segments combined in the database, five new caps, two new error codes, preparation enforced,
a strict tier that discloses less, declared default orders, forced predicates that admit null, audited
refusals, and long In lists that no longer end the process. The eleven bullets marked "Behaviour change"
change what code written for 3.0.0 does; the others fix what threw or add what was missing.
- Behaviour change. DataType.Date / DataType.DateTime resolve the member's type before building the
predicate. Every comparison on a DateTimeOffset member used to throw, and DataType.Date on any nullable
date member used to throw; both work. The null guard is emitted only where the value can be null:
IsNull / IsNotNull on a non-nullable date member of the entity answer false / true (on a DateTimeOffset
member they threw), and through a navigation they test the navigation.
- Behaviour change. A date value must be ISO 8601, year-first, or a format declared through
DwDates.Configure. A numeric day/month date such as "01/09/2026" throws the new AmbiguousDateFormat;
lenient forms such as "12:00" are InvalidFormat. The server's culture no longer decides anything.
DateTimeOffset values normalise to UTC, and a value with no zone is read as UTC. C# date objects in
Values are written year-first; a DateTime of Kind Local compared under DataType.DateTime with a
DateTimeOffset member is written with its UTC offset, so it filters on the moment it holds. Configure
refuses a declared format whose own text ISO 8601 or a year-first date already reads, such as
yyyy-MM-dd'T'HH:mm:ss'Z': declaring one could only change what such a value means, and off UTC it did.
- DateOnly members can be compared; no comparison on one worked under either date data type (IsNull and
IsNotNull did).
- HAVING on a date alias takes its type from the aggregate, so it works on DateTimeOffset.
- Security fix and behaviour change. A member named Root, It or Parent, and an alias named root, it or
parent, was read by System.Linq.Dynamic.Core as a context keyword. Root.Name and It.Name filtered,
sorted, grouped, aggregated and projected the row's own Name; Parent threw. Under ApplyPolicy that
projected [DwDenied] values, let a filter test a denied column, and applied a [DwForceWhere] scope
reached through such a navigation to the row's own column. Expressions are parsed with a library-owned
ParsingConfig with the context keywords off, and ParsingConfig.Default is no longer read. The words the
parser does keep are refused by name: a path whose first segment is new, iif, np, isnull, is, as, cast,
true, false or null, whatever the letter case, throws LogicException
FieldPath[<path>]StartsWithReservedName in every clause, guarded or not. Seven of them used to raise the
parser's ParseException, True and False an InvalidOperationException, and Null was read as the null
literal, so the query returned no rows and no error. Only a path's first segment is affected, so Owner.New
names the member. Predefined type names such as String, Math and Guid name members too, and always did.
- Behaviour change. ToListAsync(Segment) combines its condition sets into one query the database answers.
Union and Intersect combine the sets' conditions; Except removes its set's rows with NOT EXISTS on the
primary key; a type with no primary key uses SQL UNION / INTERSECT / EXCEPT. The sets used to be loaded
into lists and combined by object reference, so with AsNoTracking(), with Selects, and under ApplyPolicy
(always untracked) Intersect returned nothing, Except removed nothing and Union counted a row once per
set. Ordering, paging, projection and TotalCount now run in SQL exactly as for a Filter: text sorts by
the database's collation, Orders apply before Selects, and only the requested page is read.
- Behaviour change. PageCount on an unpaged result is 1 (0 with no rows), rather than TotalCount, or 0
for a Segment with condition sets.
- DwCaps.DefaultPageSize (default 0 = off) bounds a guarded query that sends no Page.
- Behaviour change for guarded requests. DwCaps.MaxConditionDepth (default 10) bounds how deeply
condition groups nest, and DwCaps.MaxConditionSets (default 10) how many condition sets a Segment
carries. A guarded request nested 11 levels deep, or a segment with 11 or more sets, which 3.0.0 ran,
is refused with CapExceeded unless the deployment raises the cap. Unguarded calls are not capped.
- Behaviour change for guarded requests. DwCaps.MaxConditionValues (default 1000) bounds the values one
condition carries, comparing the largest condition of the where clause, Having and every Segment set: an
In was one comparison per value for the price of one condition and one field. DwCaps.MaxAggregates
(default 50) bounds a summary's AggregateBy entries, on the Summary terminals and the composable Group and
Summary; the group floor's own count is not counted. Both refuse with CapExceeded and FieldPath "*" in
both tiers ("MaxConditionValues cap (1000), request had 1001"), refuse a value below 1, freeze with the
posture and bind from Caps:MaxConditionValues and Caps:MaxAggregates. An aggregate with no field, such as
a Count, is charged DefaultFieldCost toward MaxQueryCost; it was free. Every count cap is now checked
before any name is resolved, so an oversized request that also names an unknown field is refused with
CapExceeded, where 3.0.0 answered ConditionMustHasValidFieldName first. Unguarded calls are not
capped.
- Security fix. In, NotIn, IIn and INotIn on Text, and In and NotIn on Guid, Number and Enum, joined their
values into one flat chain, one level of expression nesting per value. EF Core and the expression compiler
walk that tree recursively, so a condition with about seven hundred values overflowed the request
thread's stack and ended the process, guarded or not; no catch can stop a stack overflow. A list longer
than 32 values is now a balanced tree of flat chains of at most 32 terms. A list of 32 or fewer is written
exactly as before, and the rows returned are the same.
- Behaviour change. ErrorCode.SelectTypeMustHaveParameterlessConstructor replaces the English sentence a
Select on an unconstructible type used to throw; LogicException gained Subject, which carries the type
name.
- Behaviour change. ApplyPolicy(ctx) refuses a context that never went through PrepareAsync, with or
without a store configured. DwPolicyContext.IsPrepared is public.
- Behaviour change. Under the Strict tier a guarded result carries no trace: result.Policy is null unless
DwPolicyOptions.IncludeTraceInResult (bool?, default null = follow the tier: off under Strict, on under
Convenience) is true. PolicyQueryable<T>.LastTrace still holds it.
- Behaviour change. Under the Strict tier an unknown field and a denied field answer alike. Outside dry
run, a path that matches nothing is refused with its clause's FieldDeniedFor* code instead of
LogicException ConditionMustHasValidFieldName; every FieldDeniedFor* refusal carries FieldPath "*" and
no RuleId or SourceOrigin, and every CapExceeded refusal carries FieldPath "*". Inside a Segment every
field refusal is FieldDeniedForSegment, and a name padded with dots is normalized as a real path is.
MaxQueryCost is checked after every field gate, so a [DwCost] weight cannot tell a hidden field from a
missing one. MissingContextValue carries FieldPath "*" and no SourceOrigin.
The trace keeps the field. The Convenience tier and dry run are unchanged.
- [DwEntity(DefaultOrder = "CreatedAt desc, Id")] orders a guarded query whose caller sends no orders,
less the fields that caller may not order by, and in a Segment less the fields denied for segments.
Unguarded calls ignore it, and so does a projected query or one that composed an Order. A field the
default keeps that is audited for Order is recorded as a use, as a caller's own order is; a field left out
is not. PolicyModelValidator reports unreadable entries, fields no query can order by and fields sealed
attributes deny for ordering as errors; unknown fields, fields only overridable attributes deny for
ordering, and fields denied for segments as warnings.
- [DwForceWhere(AllowNull = true)], ForcedPredicate.AllowNull with FromConstant / FromContext overloads
taking bool allowNull, and "allowNull" in a stored rule's forced object inject
(field op value OR field IS NULL). AllowNull on IsNull or IsNotNull is refused by the attribute, by both
factories and by a stored rule, value or no value. PolicyModelValidator now reports every malformed
[DwForceWhere] at startup; it used to surface on the first guarded query of the type.
- DwPolicyOptions.AuditRefusals (default false) writes every refused guarded query to the context's audit
buffer, under the canonical path, cut to 256 characters with control, format, line separator and
paragraph separator characters escaped. DwAuditEvent gains ErrorCode and a constructor taking it.
- StorePolicyProvider renews MaxSnapshotAge on a poll that reads back the version it serves, and that poll
clears IsDegraded. A healthy store nobody wrote to refused every guarded query one MaxSnapshotAge after
the provider last loaded it. A poll whose read was overtaken by a failed refresh or poll reloads instead.
- The composable PolicyQueryable<T>.Group applies the group floor. It went past the summary pipeline and
returned the small groups ToList(Summary) suppresses; it and the composable Summary also handed back the
floor's own __dwGroupSize column, which they no longer do.
- A forced null check built from a context key failed every guarded query on its type: the key was still
required, and its value landed on a null check that validation refuses. ForcedPredicate.FromContext now
refuses IsNull and IsNotNull and points to FromNullCheck, and a stored rule of that shape is refused when
it is read, as [DwForceWhere] already refused a ContextValue on a null check.
- The reflection cache no longer keeps an access record for a field path that fails validation. Under LRU,
the default, or LFU every invented name a caller sent stayed recorded for the life of the process.
3.0.0 Field-level policies (sections 12–32) and three companion packages. Additive for 2.x callers:
- every 2.x signature and the query engine are unchanged;
- FilterResult<T> and SummaryResult gained Policy : PolicyTrace? (null unless guarded), so JSON output
gains "policy": null;
- the core package gained Microsoft.Extensions.Configuration.Abstractions, .Configuration.Binder and
.DependencyInjection.Abstractions 6.0.0;
- PolicyException derives from LogicException, so an existing catch (LogicException) also receives
policy refusals;
- every extension method first checks [DwEntity(RequirePolicy = true)]; types without it are unaffected;
- the group floor (default 5) applies to guarded summaries only.
2.1.5 XML documentation fixes only.
2.1.4 Security fix. Condition values are escaped before they are embedded: a value ending in \ used to throw
ParseException, and a crafted value could close its literal and append predicate logic.
AggregateBy.Alias must be an identifier (AggregationMustHasValidAlias): an alias holding a comma used to
append projection terms.
2.1.3 MIT license; no API change.
2.1.2 Ordering by a path through a collection ("Tags.Value") works — Min ascending, Max descending — where it
threw "No property or field 'Value' exists in type 'List`1'". Added
OrderField[<field>]CannotEndOnCollectionOfComplexElements.
2.1.0 Condition.Values became List<object> (was List<string>): JSON callers are unaffected, C# code assigning a
List<string> no longer compiles. Values are coerced per DataType. Cache presets added.
```
### Limits by design
- `Segment` is async only. On a type with a primary key, `Except` needs a provider that translates a correlated
`EXISTS`. A type with no primary key is combined with SQL `UNION` / `INTERSECT` / `EXCEPT`: it needs the operators
the request uses, and every column to be comparable (section 6).
- `Select<T>` needs a public parameterless constructor on T. Records with only positional constructors, and
classes whose constructors all take arguments, cannot be targets; use `SelectDynamic`.
- The I-variants call `ToLower()` on both sides. On a case-sensitive collation (PostgreSQL with the `C` locale) the
database may not use an index for `LOWER(column)`; add a functional index or use the plain operators.
- Values are inlined as escaped literals, not SQL parameters. EF Core escapes them for SQL, so this is not an
injection path, but every distinct value is a distinct statement and a separate plan-cache entry.
- A path through a collection means "any element matches". There is no `All()` and no negated `Any()`.
- A member whose name is one of the parser's own words — `new`, `iif`, `np`, `isnull`, `is`, `as`, `cast`, `true`,
`false`, `null` — cannot be queried at all. No clause can name it as a path's first segment: the path is refused
with `FieldPath[<path>]StartsWithReservedName` (section 5). Rename the CLR property and map the column with
`[Column]`. Reached through a navigation (`Owner.New`) it is an ordinary member.
- A declared date format may not write text ISO 8601 or a year-first date already reads; `DwDates.Configure`
refuses one (section 4). ISO 8601 and year-first dates are read on every deployment and cannot be turned off.
- Summary rows flatten dotted group fields (`CategoryName`); `SelectDynamic` rows nest them (`Category.Name`).
- `Filter` applies `Orders` and `Page` on T before the projection, so order fields need not be selected.
- `getQueryString` needs an EF Core provider.
- Enum filtering works whether the column stores names or numbers: the parser converts the name to the enum value
before EF Core translates it. `Contains` / `StartsWith` / `EndsWith` work only on string members.
- Cache configuration changes are eventually consistent: calls already running finish with the options they read.
- With no `Selects`, a guarded query over an entity leaves out every navigation EF Core does not own once a denial
needs a projection (section 17), an included one too. Under Convenience, name the navigation in `Selects` to get
it narrowed; under Strict, name its allowed fields.
- A forced scope declared on a list's element type filters the rows that hold the list, never its elements. Selects
naming the list returns every element, those the scope excludes included, as in every release; a synthesized
projection leaves such a list out. Scope the elements where the row is built.
- A member typed `object`, a framework interface or a collection that is not generic (`IEnumerable`, `ArrayList`,
`Array`, an application's own) is opaque to the policy. It never asks for a projection; when one is needed anyway,
a projected row, a row in memory, and an entity's converted column leave it out, and an entity's other columns
keep it (what EF Core materializes itself holds no application object); and naming it
returns whatever it holds. A converted column counts as able to hold anything when its type can: `object`, a
`Dictionary<string, object>`, or a type with such a member. `BitArray`, `StringCollection`, `StringDictionary` and
`NameValueCollection` hold values. A value converter that returns an application type through a column typed
`object`, and an unmapped getter typed `object` over a private navigation, are opaque the same way: with nothing
else denied the row comes back as loaded. Type the member as what it holds.
A framework generic holding a policed type (`Dictionary<string, LineDto>`) has no paths beneath it: naming it is
refused in both tiers where the core cannot narrow it (at the top of T, or on a row in memory), narrowed away
under Convenience beneath a navigation, and a synthesized projection leaves it out.
- A projection builds the declared type. A query over the root of a hierarchy whose derived type declares a denied
field comes back as root-type rows, the derived types' allowed fields dropped too; over an abstract root the typed
terminals fail with `SelectTypeMustHaveParameterlessConstructor` and the dynamic ones return the root's members.
Query the derived type (`OfType<T>()`) to keep its fields. Subtypes are read from the assemblies loaded
when the query runs, which hold every type a row can have.
- Under a `"*"` deny, a member is kept whole only when every path beneath it the walk skips, one with no setter,
on a subtype or past four segments, is one the policy names; around a cycle it never is. Otherwise it is
narrowed, left out or refused.
- A member EF Core does not map counts as loaded and is read as its type, since its getter can hand out what EF
Core loaded: a denied field in its type asks for a projection, which leaves the member out, since the projection
cannot assign it. A getter that copies a denied column into a type with no denial is the application's to
withhold.
- Rows in memory can be any loaded subtype, and the policy does not look at the rows. When a subtype of T declares
a denied field, or a subtype or implementation of a member's type does, the rows are projected and their objects
left out, even if no row is that subtype.
- A deny-family attribute on an override, or on a member a subtype hides with `new`, denies the base path for every
subtype loaded, not only those the EF Core model maps: a view model deriving from an entity and overriding one of
its members decides the entity's own path. Declare such a class apart from the entity to keep the path open.
- Types from an unloadable `AssemblyLoadContext` stay referenced by the policy's caches, so the context is not
collected while the process runs.
- A method or property in a reshaping lambda that builds a query from captured values (a repository's query, a
specification) runs once more per guarded read, when the guard reads what it returns; one that returns a different
query on each call is enforced as it answered the guard.
- A provider that wraps EF Core's gets `AsNoTracking` only when its query's expression shows the EF Core root, as
LinqKit's and DelegateDecompiler's do; one that hides it behind its own expression runs tracking.
- A member declared as a framework collection (`IEnumerable<string>`, `List<string>`) that holds an application's own
collection class at run time is read as the framework type, so that class's own members are not read. Serializers
write only the elements.
- On EF Core 6, a reshaped query whose projection EF Core 6 cannot translate (a count over `GroupBy`/`First`, a
`SelectMany` over a captured query that builds its rows) fails guarded, where it ran unguarded; EF Core 7 and later
translate it.
- A default order reaches a projected row only through columns of the entity its `Select` reads: a projection over
an anonymous or other intermediate row takes no default.
- A simulation reads T as a source it cannot see into (section 23): every denial beneath a member counts, and a
synthesized `Clause.Selects` keeps only members holding a value.
---
## 35. Worked examples (C#)
### Usings
```csharp
using DynamicWhere.ex.Source; // extension methods
using DynamicWhere.ex.Classes.Core; // Condition ConditionGroup ConditionSet OrderBy PageBy GroupBy AggregateBy
using DynamicWhere.ex.Classes.Complex; // Filter Summary Segment
using DynamicWhere.ex.Classes.Result; // FilterResult<T> SummaryResult SegmentResult<T>
using DynamicWhere.ex.Enums; // DataType Operator Connector Direction Intersection Aggregator
using DynamicWhere.ex.Exceptions; // LogicException PolicyException
```
### Filter
```csharp
var filter = new Filter
{
ConditionGroup = new ConditionGroup
{
Connector = Connector.And,
Conditions =
{
new Condition { Sort = 1, Field = "Department", DataType = DataType.Text,
Operator = Operator.Equal, Values = { "Engineering" } },
new Condition { Sort = 2, Field = "Salary", DataType = DataType.Number,
Operator = Operator.Between, Values = { 50000, 120000 } },
},
},
Orders = new List<OrderBy> { new() { Sort = 1, Field = "HireDate", Direction = Direction.Descending } },
Page = new PageBy { PageNumber = 1, PageSize = 25 },
Selects = new List<string> { "Id", "FirstName", "Department" },
};
FilterResult<Employee> page = await db.Employees.ToListAsync(filter); // whole Employee rows, unselected members default
FilterResult<dynamic> slim = await db.Employees.ToListAsyncDynamic(filter); // rows with Id, FirstName, Department only
```
### Summary
```csharp
var summary = new Summary
{
GroupBy = new GroupBy
{
Fields = { "Department" },
AggregateBy =
{
new AggregateBy { Field = "Salary", Alias = "Total", Aggregator = Aggregator.Sumation },
new AggregateBy { Alias = "Headcount", Aggregator = Aggregator.Count },
},
},
Having = new ConditionGroup
{
Conditions = { new Condition { Sort = 1, Field = "Headcount", DataType = DataType.Number,
Operator = Operator.GreaterThanOrEqual, Values = { 3 } } },
},
Orders = new List<OrderBy> { new() { Sort = 1, Field = "Total", Direction = Direction.Descending } },
};
SummaryResult result = await db.Employees.ToListAsync(summary);
foreach (dynamic row in result.Data) Console.WriteLine($"{row.Department}: {row.Total} ({row.Headcount})");
```
### Segment
```csharp
var active = new ConditionGroup { Conditions = { new Condition { Sort = 1, Field = "IsActive", DataType = DataType.Boolean,
Operator = Operator.Equal, Values = { true } } } };
var onLeave = new ConditionGroup { Conditions = { new Condition { Sort = 1, Field = "Status", DataType = DataType.Enum,
Operator = Operator.Equal, Values = { "OnLeave" } } } };
var segment = new Segment
{
ConditionSets =
{
new ConditionSet { Sort = 1, ConditionGroup = active },
new ConditionSet { Sort = 2, Intersection = Intersection.Except, ConditionGroup = onLeave },
},
};
SegmentResult<Employee> result = await db.Employees.AsNoTracking().ToListAsync(segment); // one query
```
### An ASP.NET Core endpoint
```csharp
builder.Services.Configure<Microsoft.AspNetCore.Http.Json.JsonOptions>(o =>
o.SerializerOptions.Converters.Add(new JsonStringEnumConverter())); // accept "IContains", not only 5
app.MapPost("/employees/search", async (Filter filter, AppDbContext db) =>
{
try
{
return Results.Ok(await db.Employees.ToListAsync(filter));
}
catch (LogicException ex)
{
return Results.BadRequest(new { error = ex.Message });
}
catch (System.Linq.Dynamic.Core.Exceptions.ParseException ex)
{
return Results.BadRequest(new { error = ex.Message });
}
});
```
### A policy-protected entity
The examples above query `Employee` unguarded. Declared as below, with `RequirePolicy = true`, each of those calls
throws `PolicyRequired`: query it through `ApplyPolicy`, as the next example does.
```csharp
using DynamicWhere.ex.Policies.Attributes;
using DynamicWhere.ex.Policies.Enums;
[DwEntity(RequirePolicy = true)]
public class Employee
{
public Guid Id { get; set; }
public string FirstName { get; set; } = string.Empty;
// Confirmable, not searchable, and unreadable: the operators stop a sweep,
// the token stops the read, and [DwNoOrder] stops the sort from ranking it.
[DwAlias("Code")]
[DwOperators(Allow = new[] { Operator.Equal, Operator.In })]
[DwMask(MaskStrategy.Tokenize)]
[DwNoOrder]
public string EmployeeCode { get; set; } = string.Empty;
[DwMask(MaskStrategy.Email)]
[DwNoOrder]
public string Email { get; set; } = string.Empty;
// Rounded on the way out, aggregatable only over groups of five or more.
[DwGeneralize(GeneralizeMode.Round, Step = 5000, AllowAggregate = true, MinGroupSize = 5)]
[DwNoOrder, DwAudit, DwCost(10)]
public decimal Salary { get; set; }
// Every guarded query is scoped to this, asked for or not.
[DwForceWhere(Operator.Equal, Value = "true")]
public bool IsActive { get; set; }
// The caller's tenant, from the context.
[DwForceWhere(Operator.Equal, ContextValue = "TenantId")]
[DwNoWhere, DwNoSelect]
public int TenantId { get; set; }
[DwDenied]
public JsonDocument? WorkSchedule { get; set; }
}
```
### Wiring the policy layer
```csharp
using DynamicWhere.ex.Policies.Config;
using DynamicWhere.ex.Policies.Context;
using DynamicWhere.ex.Policies.Source;
using DynamicWhere.ex.Policies.Tokens;
// Program.cs — once
builder.Services.AddDwPolicies(builder.Configuration.GetSection("DynamicWhere:Policies"), options =>
{
options.Entities.Expose<Employee>("Employee");
options.TokenVault = new InMemoryTokenVault(); // RedisTokenVault or EfTokenVault in production
});
DwPolicy.ValidateModel(DwPolicy.Options, typeof(Employee)); // throws InvalidOperationException on any model error
// per request
app.MapPost("/employees/search", async (Filter filter, HttpContext http, AppDbContext db) =>
{
DwPolicyContext caller = await DwPolicy.PrepareAsync(new DwPolicyContext()
.WithSubject(DwSubjectKind.User, http.User.FindFirstValue(ClaimTypes.NameIdentifier)!)
.WithSubject(DwSubjectKind.Role, "Support")
.WithValue("TenantId", int.Parse(http.User.FindFirstValue("tenant_id")!)));
// with the ASP.NET Core package: DwPolicyContext caller = await http.GetPolicyContextAsync(claimsOptions);
try
{
FilterResult<Employee> result = await db.Employees.ApplyPolicy(caller).ToListAsync(filter);
return Results.Ok(result); // result.Policy: the trace; null under Strict by default
}
catch (PolicyException ex) { return Results.Json(new { error = ex.Code, field = ex.FieldPath }, statusCode: 403); }
catch (LogicException ex) { return Results.BadRequest(new { error = ex.Message }); }
});
```
It says what the library does, not what it should do
The reference is written against version 3.2.0 from the source, by hand, and checked by running the library, so it states behaviour — including the parts that are deliberately blunt, such as neither hashing nor tokenization hiding equality. Where a page in these docs disagrees with it, the file is the one to trust. For the reasoning behind a rule, the human pages carry it: start at Use cases or Security.