我正在尝试添加标题，同时从文本文件中删除某些列，因为它们包含空格，稍后在我的ETL中会引起截断问题。由于这些文件的最大大小为16GB，因此我不想将数据实际加载到R中并写回-甚至由于内存限制，甚至都不可能。
为简化易用性，已对样本数据进行了大大简化和缩小。将其复制到.txt文件"TargetTest.txt"中：
185002 ~SA ~000620~1195~1195~000~0000~Y~A~ ~S255392488
185002 ~SA ~000620~1195~1195~000~0000~Y~A~ ~S255392488
185002 ~SA ~000620~1195~1195~000~0000~Y~A~ ~S255392488
185002 ~SA ~000620~1195~1195~000~0000~Y~A~ ~S255392488
185002 ~SA ~000620~1195~1195~000~0000~Y~A~ ~S255392488
185002 ~SA ~000620~1195~1195~000~0000~Y~A~ ~S255392488
185002 ~SA ~000620~1195~1195~000~0000~Y~A~ ~S255392488
185002 ~SA ~000620~1195~1195~000~0000~Y~A~ ~S255392488
185002 ~SA ~000620~1195~1195~000~0000~Y~A~ ~S255392488
我很幸运使用command prompt从shell()运行代码。到目前为止，我的代码是：
usrHeaderNames <- c("PolicyNumber", "ReportSuffix", "ReportAccount", "PlanVariationCode", "ReportCode", "FranchiseCodeOne", "FranchiseCodeTwo", "UNETRegionCode", "FinancialArrangementIndicator", "Filler10", "EmployeeID", "MemberLastName", "EmployeeSex", "EmployeeDateofBirth", "EmployeeZIPCode", "EmployeeStatus", "Filler17", "DependentNumber", "IndividualID", "MemberRelationshipCode", "MemberFirstName", "MemberDateofBirth", "MemberSex", "MedicareEligibilityIndicator", "MemberMarket", "PatientNumber", "EmployeePOSInOutofAreaInd.", "EmployeePPOInOutofAreaInd.", "MemberProductCode", "Filler30", "PHIIndicator", "Filler32", "ClaimReferenceNumber", "DateProcessed", "ElectronicBillingIndicator", "ClaimsOfficeNumber", "TransactionCode", "DateClaimReceived", "ClaimAdjusterNumber", "ProcessingOfficeNumber", "UniqueCheckIdentifier", "TransactionType", "StateTaxEligibilityIndicator", "DocumentControlSerialNumber", "FilmingOfficeNumber", "ProviderType", "ProviderFullName", "ProviderTaxIDPrefix", "ProviderTaxID", "ProviderTaxIDSuffix", "ProviderIPA", "PremiumProviderDerivedBenefitTierLevelIndicator", "ProviderZIPCode", "ProviderSpecialtyCode", "PremiumProviderIndicator", "ProviderMarket", "MPIN", "ProviderNetworkParticipatingInd.", "CoveringPhysicianIndicator", "Filler60", "NationalDrugCode", "Filler62", "CauseCode", "DischargeStatus", "Filler65", "PlaceofService", "ServiceCode", "ServiceCodeModifier", "ProcedureModifier2", "DateofServiceFrom", "DateofServiceTo", "ServiceCount", "Filler73", "CapitatedEncounterIndicator", "HospitalDRG", "Filler76", "BilledAmount", "NotCoveredAmount", "RemarkCode", "ChargeLevelRemarkCode", "Filler81", "ReconsideredNotCoveredAmount", "ReconsiderationRemarkCode", "ClaimLevelRemarkCode", "Filler85", "BenefitsLimitations", "DiscountAmount", "DiscountType", "ProviderContractType", "AllowableExpense", "Deductible", "Copay", "Coinsurance", "GrossBenefitsPayable", "OtherInsuranceAmount", "OtherInsuranceIndicator", "OtherInsuranceType", "MiscellaneousReductionsAmount", "NetPaid", "BenefitPlanComplianceIndicator", "PayeeType", "TaxRecordIndicator", "Out-of-PocketOffsetAmount", "ClaimStatusCode", "OverrideCode", "ServiceOrder", "PayoutSummaryCategory", "Filler108", "CheckSuppressionIndicator", "PCPTaxIDPrefix", "PCPTaxID", "PCPTaxIDSuffix", "ProviderClassificationCode", "RevenueCode-1", "RevenueCode-2", "RevenueCode-3", "RevenueCode-4", "RevenueCode-5", "RevenueCode-6", "RevenueCode-7", "Fillerreservedarea", "HRAAmount", "NPINumber", "PrimaryDiagnosis", "SecondaryDiagnosis", "TertiaryDiagnosis", "ICD-10INDICATOR", "RevenueCode-8", "RevenueCode-9", "RevenueCode-10", "RevenueCode-11", "RevenueCode-12", "RevenueCode-13", "RevenueCode-14", "RevenueCode-15", "RevenueCode-16", "RevenueCode-17", "RevenueCode-18", "RevenueCode-19", "RevenueCode-20", "RevenueCodeCount1", "RevenueCodeCount2", "RevenueCodeCount3", "RevenueCodeCount4", "RevenueCodeCount5", "RevenueCodeCount6", "RevenueCodeCount7", "RevenueCodeCount8", "RevenueCodeCount9", "RevenueCodeCount10", "RevenueCodeCount11", "RevenueCodeCount12", "RevenueCodeCount13", "RevenueCodeCount14", "RevenueCodeCount15", "RevenueCodeCount16", "RevenueCodeCount17", "RevenueCodeCount18", "RevenueCodeCount19", "RevenueCodeCount20", "RevenueSourceChargeAmt1", "RevenueSourceChargeAmt2", "RevenueSourceChargeAmt3", "RevenueSourceChargeAmt4", "RevenueSourceChargeAmt5", "RevenueSourceChargeAmt6", "RevenueSourceChargeAmt7", "RevenueSourceChargeAmt8", "RevenueSourceChargeAmt9", "RevenueSourceChargeAmt10", "RevenueSourceChargeAmt11", "RevenueSourceChargeAmt12", "RevenueSourceChargeAmt13", "RevenueSourceChargeAmt14", "RevenueSourceChargeAmt15", "RevenueSourceChargeAmt16", "RevenueSourceChargeAmt17", "RevenueSourceChargeAmt18", "RevenueSourceChargeAmt19", "RevenueSourceChargeAmt20", "PrimarySurgicalProcedureCode", "SecondarySurgicalProcedureCode", "TertiarySurgicalProcedureCode", "RESERVED")
usrCompleteFolderPath <- "C:/Users/pboswell/Downloads/"
usrColumnIgnore <- c("1", "1", "1", "0", "1", "1", "1", "1", "1", "1", "0", "0", "1", "1", "0", "1", "1", "1", "1", "0", "0", "0", "0", "1", "1", "0", "1", "1", "1", "1", "1", "1", "0", "1", "1", "1", "1", "0", "1", "1", "1", "1", "1", "1", "1", "0", "0", "1", "0", "1", "1", "1", "0", "0", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "0", "0", "0", "0", "0", "0", "1", "1", "1", "0", "1", "0", "1", "1", "1", "1", "1", "1", "1", "1", "1", "0", "0", "1", "0", "0", "0", "0", "1", "0", "0", "1", "1", "0", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "0", "0", "0", "0", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1")
usrWorkingFileName <- "TargetTest.txt"
vcColumnIndex <- which(usrColumnIgnore==0)
vcDelimiter <- "~"
vcOuputFileName <- "TargetTestNew.txt"
cmdScript <- paste0(
"cd ",gsub("/","\\\\",usrCompleteFolderPath)
," && "
,"echo ",paste0(usrHeaderNames[vcColumnIndex],collapse=vcDelimiter)," > ",vcOuputFileName
," && "
,"for /f \"tokens=",paste0(vcColumnIndex,collapse=",")," delims=~\" %1 in (",usrWorkingFileName,") DO echo ",paste0("%",paste0(seq.int(vcColumnIndex),collapse=paste0(vcDelimiter,"%")))," >> ",vcOuputFileName
)
对于标题文件，我能够使用echo ____ > ____轻松创建标题文件：
shell(paste0("echo ",paste0(usrHeaderNames[vcColumnIndex],collapse=vcDelimiter)," > ",vcOuputFileName))
，然后使用type ____ >> ____附加实际数据：
shell(paste0("type ",usrWorkingFileName," >> ",vcOuputFileName))
但是我想我可以结合这些步骤，并使用FOR /F ["options"] %%parameter IN ("Text string to process") DO command方法将所需的列简单地附加到头文件中：
shell(paste0("for /f \"tokens=",paste0(vcColumnIndex,collapse=",")," delims=~\" %1 in (",usrWorkingFileName,") DO echo ",paste0("%",paste0(seq.int(vcColumnIndex),collapse=paste0(vcDelimiter,"%")))," >> ",vcOuputFileName))
但是这使用了for循环，这在大型数据集上非常糟糕。原来的标头/追加方法在500MB上花了2分钟，而新的for循环方法在30分钟后仍未完成（我取消了该过程）。
1）我必须在Windows中为此使用for循环吗？
2）Linux awk或cut命令是否工作得更好（即分批）？如果是这样，我可以使用Windows的端口来执行R代码吗？
3）是否可以使用另一种方法来完成此操作，例如在写入文件后查找并替换空白？