Extract SqlBulkCopy column names using dynamic SQL by paulmedynski · Pull Request #4092 · dotnet/SqlClient

paulmedynski · 2026-03-26T13:33:25Z

Description

#3590 allowed SqlBulkCopy to find and work with hidden columns (even if those columns weren't accessible server-side.) It contained a check on the all_columns DMV to make sure that we only checked the graph_type column when querying if that column existed; this was to maintain SQL 2016 compatibility.

#3714 highlighted that both queries are compiled and fail at the point of compilation anyway, so this wasn't accomplishing anything. To make this work, we need to use dynamic SQL to run the column queries. This PR does so.

Supersedes #3719. Original PR by @edwardneal. This PR was created as a direct branch of the SqlClient repo so I can run the new SQL 16 and 17 CI pipeline stages manually to prove this fixes the issue.

Issues

Fixes #3714.

Testing

Manual testing of all test cases against a SQL 2016 instance (via the updated CI pipelines).

Copilot

Pull request overview

This PR updates SqlBulkCopy’s target-column discovery query to use dynamic SQL so that referencing sys.all_columns.graph_type doesn’t cause compilation failures on SQL Server 2016, addressing regression #3714.

Changes:

Reworks SqlBulkCopy.CreateInitialQuery() to build and execute the sys.all_columns column-name query via sp_executesql.
Adjusts manual test SQL statistics expectations to account for the additional statements executed.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/SqlBulkCopyTest/CopyAllFromReader.cs	Updates expected `SelectCount` / `SelectRows` due to extra statements from dynamic SQL.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopy.cs	Uses dynamic SQL for column-name extraction to avoid SQL 2016 compilation errors when `graph_type` is absent.

codecov · 2026-03-26T14:39:19Z

Codecov Report

❌ Patch coverage is 63.63636% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.51%. Comparing base (60d4b92) to head (8074cd5).
⚠️ Report is 17 commits behind head on main.

Files with missing lines	Patch %	Lines
...Client/src/Microsoft/Data/SqlClient/SqlBulkCopy.cs	63.63%	4 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (60d4b92) and HEAD (8074cd5). Click for more details.

HEAD has 1 upload less than BASE

Flag BASE (60d4b92) HEAD (8074cd5)

CI-SqlClient 1 0

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4092      +/-   ##
==========================================
- Coverage   73.22%   66.51%   -6.72%     
==========================================
  Files         280      274       -6     
  Lines       43000    65785   +22785     
==========================================
+ Hits        31486    43754   +12268     
- Misses      11514    22031   +10517

Flag	Coverage Δ
CI-SqlClient	`?`
PR-SqlClient-Project	`66.51% <63.63%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

paulmedynski · 2026-03-27T11:21:21Z

/azp run

azure-pipelines · 2026-03-27T11:21:48Z

Azure Pipelines successfully started running 2 pipeline(s).

priyankatiwari08

Looks good to me, Just added a suggestion to add comment related to explaining why CatalogName is interpolated directly into the dynamic SQL string rather than parameterized

paulmedynski · 2026-04-01T10:34:20Z

Addressed both open review comments in 70269ce:

SQLServerVersion lazy-init fix (Copilot): Changed s_sQLServerVersion field initializer from string.Empty to null (and renamed to s_sqlServerVersion) so the ??= null-coalescing assignment actually triggers. IsAtLeastSQL2017() / IsAtLeastSQL2019() will now correctly query the server version on first access.
CatalogName interpolation comment (@priyankatiwari08): Added an inline comment block explaining that CatalogName and escapedObjectName are interpolated directly because SQL Server doesn't allow parameterizing identifiers, and both values are pre-escaped via SqlServerEscapeHelper.

This enables the query to compile correctly when the graph_type column is missing

Updated [DotNetEnv](https://github.com/tonerdo/dotnet-env) from 3.1.1 to 3.2.0. <details> <summary>Release notes</summary> _Sourced from [DotNetEnv's releases](https://github.com/tonerdo/dotnet-env/releases)._ ## 3.2.0 - Switch parsing to Superpower (from Sprache) - Fix utf8 parsing - Interpolated variables parsing Commits viewable in [compare view](tonerdo/dotnet-env@v3.1.1...v3.2.0). </details> Updated [Microsoft.AspNetCore.Authentication.Google](https://github.com/dotnet/dotnet) from 10.0.6 to 10.0.7. <details> <summary>Release notes</summary> _Sourced from [Microsoft.AspNetCore.Authentication.Google's releases](https://github.com/dotnet/dotnet/releases)._ No release notes found for this version range. Commits viewable in [compare view](https://github.com/dotnet/dotnet/commits). </details> Updated [Microsoft.Data.SqlClient](https://github.com/dotnet/sqlclient) from 7.0.0 to 7.0.1. <details> <summary>Release notes</summary> _Sourced from [Microsoft.Data.SqlClient's releases](https://github.com/dotnet/sqlclient/releases)._ ## 7.0.1 This update brings the following changes since the [7.0.0](https://github.com/dotnet/SqlClient/blob/release/7.0/release-notes/7.0/7.0.0.md) release: ### Fixed - Fixed `SqlBulkCopy` failing on SQL Server 2016 with `Invalid column name 'graph_type'` error. The column metadata query now uses dynamic SQL so that references to the `graph_type` column (introduced in SQL Server 2017) are not compiled on older versions that lack the column. ([#3714](dotnet/SqlClient#3714), [#4092](dotnet/SqlClient#4092), [#4147](dotnet/SqlClient#4147)) - Fixed `SqlBulkCopy` failing on Azure Synapse Analytics dedicated SQL pools. The column-list query previously used a variable-assignment pattern that Synapse does not support; it now uses `STRING_AGG` when targeting Synapse (engine edition 6) and falls back to the variable-assignment approach for SQL Server 2016 compatibility. ([#4149](dotnet/SqlClient#4149), [#4176](dotnet/SqlClient#4176), [#4182](dotnet/SqlClient#4182)) - Fixed `SqlDataReader.GetFieldType()` and `GetProviderSpecificFieldType()` returning `typeof(byte[])` instead of `typeof(SqlVector<float>)` for vector float32 columns. The methods now follow the same type-determination logic as `GetValue()`. ([#4104](dotnet/SqlClient#4104), [#4105](dotnet/SqlClient#4105), [#4152](dotnet/SqlClient#4152)) - Added missing `System.Data.Common` (v4.3.0) NuGet package dependency for .NET Framework consumers. The inbox `System.Data.Common` assembly on .NET Framework predates APIs such as `IDbColumnSchemaGenerator`; without the explicit NuGet dependency, consumers encountered `CS0012` compilation errors when using these types through `Microsoft.Data.SqlClient`. ([#4063](dotnet/SqlClient#4063), [#4074](dotnet/SqlClient#4074)) ### Changed - Enabled the User Agent TDS feature extension unconditionally. The `Switch.Microsoft.Data.SqlClient.EnableUserAgent` AppContext switch has been removed; the driver now always sends User Agent information during login. ([#4124](dotnet/SqlClient#4124), [#4154](dotnet/SqlClient#4154)) - Added type forwards from the core `Microsoft.Data.SqlClient` assembly to public types that were moved to the `Microsoft.Data.SqlClient.Extensions.Abstractions` package: `SqlAuthenticationMethod`, `SqlAuthenticationParameters`, `SqlAuthenticationProvider`, `SqlAuthenticationProviderException`, and `SqlAuthenticationToken`. This ensures binary compatibility for assemblies compiled against earlier versions of `Microsoft.Data.SqlClient` where these types lived in the core assembly. ([#4067](dotnet/SqlClient#4067), [#4117](dotnet/SqlClient#4117)) - Fixed API documentation include paths and duplicate doc snippets. ([#4084](dotnet/SqlClient#4084), [#4086](dotnet/SqlClient#4086), [#4107](dotnet/SqlClient#4107), [#4161](dotnet/SqlClient#4161)) ## Contributors We thank the following public contributors. Their efforts toward this project are very much appreciated. - [edwardneal](https://github.com/edwardneal) ## Target Platform Support - .NET Framework 4.6.2+ (Windows x86, Windows x64, Windows ARM64) - .NET 8.0+ (Windows x86, Windows x64, Windows ARM, Windows ARM64, Linux, macOS) ### Dependencies #### .NET 9.0 - Microsoft.Bcl.Cryptography 9.0.13 - Microsoft.Data.SqlClient.Extensions.Abstractions 1.0.0 - Microsoft.Data.SqlClient.Internal.Logging 1.0.0 - Microsoft.Data.SqlClient.SNI.runtime 6.0.2 - Microsoft.Extensions.Caching.Memory 9.0.13 - Microsoft.IdentityModel.JsonWebTokens 8.16.0 - Microsoft.IdentityModel.Protocols.OpenIdConnect 8.16.0 - Microsoft.SqlServer.Server 1.0.0 - System.Configuration.ConfigurationManager 9.0.13 - System.Security.Cryptography.Pkcs 9.0.13 #### .NET 8.0 - Microsoft.Bcl.Cryptography 8.0.0 - Microsoft.Data.SqlClient.Extensions.Abstractions 1.0.0 ... (truncated) Commits viewable in [compare view](dotnet/SqlClient@v7.0.0...v7.0.1). </details> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

paulmedynski requested a review from a team as a code owner March 26, 2026 13:33

paulmedynski added this to the 7.1.0-preview1 milestone Mar 26, 2026

paulmedynski added the Regression 💥 Issues that are regressions introduced from earlier PRs. label Mar 26, 2026

Copilot AI review requested due to automatic review settings March 26, 2026 13:33

github-project-automation Bot added this to SqlClient Board Mar 26, 2026

github-project-automation Bot moved this to To triage in SqlClient Board Mar 26, 2026

paulmedynski mentioned this pull request Mar 26, 2026

Fix 3714 | Extract SqlBulkCopy column names using dynamic SQL #3719

Closed

Copilot started reviewing on behalf of paulmedynski March 26, 2026 13:34 View session

paulmedynski mentioned this pull request Mar 26, 2026

SQL Bulk Copy Fails on SQL Server 2016 / 130 #3714

Closed

paulmedynski moved this from To triage to In review in SqlClient Board Mar 26, 2026

Copilot AI reviewed Mar 26, 2026

View reviewed changes

Comment thread src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopy.cs

Comment thread src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopy.cs

paulmedynski commented Mar 26, 2026

View reviewed changes

Comment thread src/Microsoft.Data.SqlClient/tests/ManualTests/DataCommon/DataTestUtility.cs

Comment thread src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/SqlBulkCopyTest/SqlGraphTables.cs

Copilot AI review requested due to automatic review settings March 26, 2026 17:38

paulmedynski force-pushed the dev/paul/3714-sqlbulkcopy-sql2016 branch from 5ef395f to 1be29d6 Compare March 26, 2026 17:38

Copilot started reviewing on behalf of paulmedynski March 26, 2026 17:39 View session

Copilot AI reviewed Mar 26, 2026

View reviewed changes

Comment thread src/Microsoft.Data.SqlClient/tests/ManualTests/DataCommon/DataTestUtility.cs

cheenamalhotra previously approved these changes Mar 26, 2026

View reviewed changes

cheenamalhotra added the Hotfix Candidate 🚑 Issues/PRs that are candidate for backporting to earlier supported versions. label Mar 28, 2026

priyankatiwari08 reviewed Mar 30, 2026

View reviewed changes

Comment thread src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopy.cs

priyankatiwari08 previously approved these changes Mar 30, 2026

View reviewed changes

paulmedynski added Hotfix 7.0.1 When this PR merges, automatically open a PR to cherry-pick to the 7.0.1 branch and removed Hotfix Candidate 🚑 Issues/PRs that are candidate for backporting to earlier supported versions. labels Mar 31, 2026

paulmedynski dismissed stale reviews from priyankatiwari08 and cheenamalhotra via 70269ce April 1, 2026 10:32

cheenamalhotra previously approved these changes Apr 2, 2026

View reviewed changes

Build column name query using dynamic SQL

49ea581

This enables the query to compile correctly when the graph_type column is missing

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extract SqlBulkCopy column names using dynamic SQL#4092

Extract SqlBulkCopy column names using dynamic SQL#4092
paulmedynski merged 7 commits intomainfrom
dev/paul/3714-sqlbulkcopy-sql2016

paulmedynski commented Mar 26, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Mar 26, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

paulmedynski commented Mar 27, 2026

Uh oh!

azure-pipelines Bot commented Mar 27, 2026

Uh oh!

Uh oh!

priyankatiwari08 left a comment

Uh oh!

paulmedynski commented Apr 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

paulmedynski commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Issues

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

paulmedynski commented Mar 27, 2026

Uh oh!

azure-pipelines Bot commented Mar 27, 2026

Uh oh!

Uh oh!

priyankatiwari08 left a comment

Choose a reason for hiding this comment

Uh oh!

paulmedynski commented Apr 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

paulmedynski commented Mar 26, 2026 •

edited

Loading

codecov Bot commented Mar 26, 2026 •

edited

Loading