在向下滚动RecyclerView后，SwipeRefreshLayout不可见

Question

我的布局文件

<android.support.v4.widget.SwipeRefreshLayout
    android:id="@+id/swipeRefreshLayoutColleague"
    android:layout_below="@+id/main_header_layout"
    android:layout_width="match_parent"
    android:layout_height="match_parent">
    <com.ms.engage.utils.recycler.EmptyRecyclerView
        android:id="@+id/colleagues_list"
        android:layout_width="match_parent"
        android:layout_height="match_parent"/>
</android.support.v4.widget.SwipeRefreshLayout> 

码

swipeRefreshLayout=(SwipeRefreshLayout)findViewById(R.id.swipeRefreshLayoutColleague);
swipeRefreshLayout.setColorSchemeResources(R.color.pull_to_refresh);
swipeRefreshLayout.setOnRefreshListener(this);