[CONNECTORS-1284] Not all SharePoint Metadata Fields are returned

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: ManifoldCF 2.4
• Component/s: SharePoint connector

Environment

ManifoldCF 2.3, SharePoint 2013

• Assignee: Karl Wright
• Reporter: Markus Schuch
• Created: 07/Mar/16 10:18
• Updated: 07/Mar/16 14:23
• Resolved: 07/Mar/16 14:23

Description

Running ManifoldCF 2.3, Sharepoint Connector against SP 2013.

Active Metadata Rule:

metadatarule: {
_value_: "",
_attribute_match: "/*",
_attribute_action: "include",
_attribute_allmetadata: "true"
}

We miss this field:
WorkingAreaID_x003a_SubProcesses with the value WeitererTest
which was returned by the connector on MCF 1.7.1.

We noted the Change from CONNECTOR-1203 and we expect the field to come over by a different name. But it is not there at all.

Then we started to experiment. When changing

            for (String field : fieldNames.values())
            {
              fields[j++] = field;
            }         

to

            for (String field : fieldNames.keySet())
            {
              fields[j++] = field;
            }      

(At lines 857, 1309 in the class SharePointRepository)

the field is returned.

Here comes the debug output for both cases:

DEBUG output for getFieldValues on 2.3:

2016-03-07 10:37:25,021 DEBUG [Worker thread '4'] org.apache.manifoldcf.connectors: SharePoint: Processing file '/workingareas/weiterertest/Documents//Testdatei.txt'; url: 'https://m10005.sharepoint-tu.noncd-test.rz.test.de/workingareas/weiterertest/Documents/Testdatei.txt'

2016-03-07 10:37:25,022 DEBUG [Worker thread '4'] org.apache.manifoldcf.connectors: SharePoint: In getFieldValues; fieldNames=[App Created By, App Modified By, Approval Status, Approve Folder WF, Approver Comments, Author, Auto Approval, Check In Comment, Checked Out To, Content Type, Copy Source, Countries, Created, Created By, Document Approval, Document Status, Edit, File Size, Folder Child Count, ID, Item Child Count, Local Entities, Modified, Modified By, Name, Publication Date, SCC, Scheduling End Date, Scheduling Start Date, Scope, SetMetaDataWA_ApDoc, SetMetaDataWA_NonApDoc, Source Name (Converted Document), Source Version (Converted Document), SubProcesses, Title, Type, Version, WorkingArea, WorkingArea ID], site='/workingareas/weiterertest', docLibrary='{64262471-2E4E-4E40-9A94-BDC03DADECCD}', docId='/Documents/Testdatei.txt', dspStsWorks=false

2016-03-07 10:37:25,524 DEBUG [Worker thread '4'] org.apache.manifoldcf.connectors: SharePoint: getListItems FileRef value 'workingareas/weiterertest/Documents/Testdatei.txt', xml response: '<ns1:listitems xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" xmlns:ns1="http://schemas.microsoft.com/sharepoint/soap/">
<rs:data ItemCount="1">
   <z:row ows_Author="131;#[e-mail]" ows_Countries="1;#sdfsdf" ows_Created="2015-12-11 13:30:33" ows_Edit="0" ows_ID="1" ows_Modified="2015-12-11 13:42:14" ows_Title="sdf" ows__Level="2" ows_ProgId="1;#" ows_Local_x0020_Entities="1;#sdfsdf" ows_Last_x0020_Modified="1;#2015-12-11 13:42:13" ows_owshiddenversion="7" ows_FileLeafRef="1;#Testdatei.txt" ows_PermMask="0x7fffffffffffffff" ows_FileRef="1;#workingareas/weiterertest/Documents/Testdatei.txt" ows__ModerationStatus="3" ows_UniqueId="1;#{00105529-BD03-4408-B509-A1EDF167752A}" ows_FSObjType="1;#0" ows_Created_x0020_Date="1;#2015-12-11 13:30:33" ows_WorkingAreaID="22;#22" ows_DocIcon="txt" ows_Editor="131;#[e-mail]"/>
</rs:data>
</ns1:listitems>'

DEBUG output for getFieldValues on our patched version

2016-03-07 10:31:02,016 DEBUG [Worker thread '22'] org.apache.manifoldcf.connectors: SharePoint: Processing file '/workingareas/weiterertest/Documents//Testdatei.txt'; url: 'https://m10005.sharepoint-tu.noncd-test.rz.test.de/workingareas/weiterertest/Documents/Testdatei.txt'

2016-03-07 10:31:02,016 DEBUG [Worker thread '22'] org.apache.manifoldcf.connectors: SharePoint: In getFieldValues; fieldNames=[AppAuthor, AppEditor, ApproveF, Author, Auto_x0020_Approval, CheckoutUser, ContentType, Countries, Created, DocIcon, DocumentStatus, Document_x0020_Approval, Document_x0020_Approval_x0028_1_x0029_, Edit, Editor, FileLeafRef, FileSizeDisplay, FolderChildCount, ID, ItemChildCount, LinkFilename, LinkFilenameNoMenu, Local_x0020_Entities, Modified, ParentLeafName, ParentVersionString, PublicationDate, PublishingExpirationDate, PublishingStartDate, ScopeDocument, SetMetaD, SetMetaD0, Title, WorkingAreaID, WorkingAreaID_x003a_Name, WorkingAreaID_x003a_SCC, WorkingAreaID_x003a_SubProcesses, _Author, _CheckinComment, _CopySource, _ModerationComments, _ModerationStatus, _UIVersionString], site='/workingareas/weiterertest', docLibrary='{64262471-2E4E-4E40-9A94-BDC03DADECCD}', docId='/Documents/Testdatei.txt', dspStsWorks=false

2016-03-07 10:31:02,688 DEBUG [Worker thread '22'] org.apache.manifoldcf.connectors: SharePoint: getListItems FileRef value 'workingareas/weiterertest/Documents/Testdatei.txt', xml response: '<ns1:listitems xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" xmlns:ns1="http://schemas.microsoft.com/sharepoint/soap/">
<rs:data ItemCount="1">
   <z:row ows_AppEditor="22;#Workflow" ows_Author="131;#[e-mail]" ows_ContentType="Approval Document" ows_Countries="1;#sdfsdf" ows_Created="2015-12-11 13:30:33" ows_DocIcon="txt" ows_DocumentStatus="Ready for Review" ows_Edit="0" ows_Editor="131;#[e-mail]" ows_FileLeafRef="1;#Testdatei.txt" ows_FileSizeDisplay="27" ows_FolderChildCount="1;#0" ows_ID="1" ows_ItemChildCount="1;#0" ows_LinkFilename="Testdatei.txt" ows_LinkFilenameNoMenu="Testdatei.txt" ows_Local_x0020_Entities="1;#sdfsdf" ows_Modified="2015-12-11 13:42:14" ows_ParentLeafName="1;#" ows_ParentVersionString="1;#" ows_Title="sdf" ows_WorkingAreaID="22;#22" ows_WorkingAreaID_x003a_Name="22;#WeitererTest" ows_WorkingAreaID_x003a_SCC="22;#Amerika" ows_WorkingAreaID_x003a_SubProcesses="22;#AP" ows__Author="sdf" ows__CheckinComment="1;#" ows__ModerationStatus="3" ows__UIVersionString="0.5" ows__Level="2" ows_ProgId="1;#" ows_Last_x0020_Modified="1;#2015-12-11 13:42:13" ows_owshiddenversion="7" ows_PermMask="0x7fffffffffffffff" ows_FileRef="1;#workingareas/weiterertest/Documents/Testdatei.txt" ows_UniqueId="1;#{00105529-BD03-4408-B509-A1EDF167752A}" ows_FSObjType="1;#0" ows_Created_x0020_Date="1;#2015-12-11 13:30:33"/>
</rs:data>
</ns1:listitems>'

Issue Links

• is related to: CONNECTORS-1203 Erratic handling of Sharepoint 2010 _ModerationStatus metadata